On Fri, Dec 08, 2017 at 11:14:16AM +0100, Peter Zijlstra wrote: > On Thu, Dec 07, 2017 at 09:56:57PM +0100, Daniel Vetter wrote: > > On Thu, Dec 07, 2017 at 08:57:09PM +0100, Peter Zijlstra wrote: > > > > Is what it says I suppose. Now I don't know enough about that i915 code > > > to say if that breadcrumbs_signal thread can ever trigger a fault or > > > not. I got properly lost in that dma_fence callback maze. > > > > > > You're saying not? > > > > Our own kthread, no. At least a tons of run on our CI with the kthread > > patch applied shut up lockdep splats for good. And since we have all the > > i915 kthreads still with the same lockdep_map even with the patch applied, > > since they are all created in the same function, I think that's pretty > > solid evidence. > > > > [There's also really no reasonable reason for it to fault, but I trust > > automated tools more to check this stuff than my own brain. The test suite > > we're running is fairly nasty and does all kinds of corner case > > thrashing. Note that the dma_fence callbacks can be provideded by any > > other driver (think multi-gpu desktops and stuff), but the contract is > > that they must be able to handle hardirq context. Faulting's definitely > > not on the table.] > > OK, good. Aside: Could/should we take some fake lockdep locks around these callbacks, since not all drivers call them from a hardirq context? Just to validate that everyone follows the contract. I need to ponder proper lockdep annotations for dma_fence anyway, since they're just completions which also have some support for direct hw->hw signalling. > > The problem lockdep seems to complain about is that some random other > > kthread could fault, end up in the i915 fault handler, and get stuck until > > i915_reset_device is done doing what it needs to do. But as long as that > > kthread is in turn not providing a service that i915_reset_device needs, I > > don't see how that can deadlock. And if we have that case (there was > > definitely plenty of that stuff that cross-release uncovered in our code, > > we had to shuffle a bunch of allocations and things out from under > > dev->struct_mutex), then there should be another lock or completion that > > closes the loop again. > > Indeed so. > > > > (also, that comment near need_resched() doesn't make sense to me) > > > > I assume you mean the one in intel_breadcrumbs_signaler(). The hw design > > is somewhat screwed up and depends upon ridiculously low interrupt > > servicing time. We get there by essentially implementing something like > > netdev polled mode, from irq context. Like net polling if the scheduler > > gets pissed at us we stop and dump it all into a kthread. From a latency > > and ops/sec pov a gpu is pretty close to networking sometimes. > > > > [Note: I just have a rough idea what the code is supposed to do, I didn't > > write/review/debug that one.] > > The thing is though; that calling schedule() from an RT thread doesn't > help anything if it goes running instantly again. > > And looking more; that uses the waitqueue code 'creatively' it doesn't > actually have a condition to wait on, so wtf is it doing with a > waitqueue? Yeah that looks fishy. I discussed it with Chris, and the waitqueue stuff is indeed broken. Chris has a patch to address that. The main wakeup logic of the thread is done through wake_up_process(breadcrumb->signaller) directly, so no ordering issue wrt adding to the waitqueue. And then the proper waker/wakee pattern holds: - set tast state - check condition - schedule() Link to Chris' patch: https://www.spinics.net/lists/intel-gfx/msg149891.html Thanks, Daniel -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ dri-devel mailing list dri-devel@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/dri-devel
