On Wed, Jul 22, 2020 at 3:12 PM Thomas Hellström (Intel) <thomas_os@xxxxxxxxxxxx> wrote: > On 2020-07-22 14:41, Daniel Vetter wrote: > > Ah I think I misunderstood which options you want to compare here. I'm > > not sure how much pain fixing up "dma-fence as memory fence" really > > is. That's kinda why I want a lot more testing on my annotation > > patches, to figure that out. Not much feedback aside from amdgpu and > > intel, and those two drivers pretty much need to sort out their memory > > fence issues anyway (because of userptr and stuff like that). > > > > The only other issues outside of these two drivers I'm aware of: > > - various scheduler drivers doing allocations in the drm/scheduler > > critical section. Since all arm-soc drivers have a mildly shoddy > > memory model of "we just pin everything" they don't really have to > > deal with this. So we might just declare arm as a platform broken and > > not taint the dma-fence critical sections with fs_reclaim. Otoh we > > need to fix this for drm/scheduler anyway, I think best option would > > be to have a mempool for hw fences in the scheduler itself, and at > > that point fixing the other drivers shouldn't be too onerous. > > > > - vmwgfx doing a dma_resv in the atomic commit tail. Entirely > > orthogonal to the entire memory fence discussion. > > With vmwgfx there is another issue that is hit when the gpu signals an > error. At that point the batch might be restarted with a new meta > command buffer that needs to be allocated out of a dma pool. in the > fence critical section. That's probably a bit nasty to fix, but not > impossible. Yeah reset is fun. From what I've seen this isn't any worse than the hw allocation issue for drm/scheduler drivers, they just allocate another hw fence with all that drags along. So the same mempool should be sufficient. The really nasty thing around reset is display interactions, because you just can't take drm_modeset_lock. amdgpu fixed that now (at least the modeset_lock side, not yet the memory allocations that brings along). i915 has the same problem for gen2/3 (so really old stuff), and we've solved that by breaking&restarting all i915 fence waits, but that predates multi-gpu and wont work for shared fences ofc. But it's so old and predates all multi-gpu laptops that I think wontfix is the right take. Other drm/scheduler drivers don't have that problem since they're all render-only, so no display driver interaction. > > I'm pretty sure there's more bugs, I just haven't heard from them yet. > > Also due to the opt-in nature of dma-fence we can limit the scope of > > what we fix fairly naturally, just don't put them where no one cares > > :-) Of course that also hides general locking issues in dma_fence > > signalling code, but well *shrug*. > Hmm, yes. Another potential big problem would be drivers that want to > use gpu page faults in the dma-fence critical sections with the > batch-based programming model. Yeah that's a massive can of worms. But luckily there's no such driver merged in upstream, so hopefully we can think about all the constraints and how to best annotate&enforce this before we land any code and have big regrets. -Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
