On Tue, Oct 06, 2020 at 07:24:30PM +0200, Daniel Vetter wrote: > On Tue, Oct 6, 2020 at 6:34 PM Daniel Vetter <daniel@xxxxxxxx> wrote: > > > > On Tue, Oct 06, 2020 at 12:49:56PM -0300, Jason Gunthorpe wrote: > > > On Tue, Oct 06, 2020 at 11:22:14AM +0200, Daniel Vetter wrote: > > > > > > > > For reinstanting the pages you need: > > > > > > > > - dma_resv_lock, this prevents anyone else from issuing new moves or > > > > anything like that > > > > - dma_resv_get_excl + dma_fence_wait to wait for any pending moves to > > > > finish. gpus generally don't wait on the cpu, but block the dependent > > > > dma operations from being scheduled until that fence fired. But for rdma > > > > odp I think you need the cpu wait in your worker here. > > > > > > Reinstating is not really any different that the first insertion, so > > > then all this should be needed in every case? > > > > Yes. Without move_notify we pin the dma-buf into system memory, so it > > can't move, and hence you also don't have to chase it. But with > > move_notify this all becomes possible. > > I just realized I got it wrong compared to gpus. I needs to be: > 1. dma_resv_lock > 2. dma_buf_map_attachment, which might have to move the buffer around > again if you're unlucky > 3. wait for the exclusive fence > 4. put sgt into your rdma ptes > 5 dma_resv_unlock > > Maybe also something we should document somewhere for dynamic buffers. > Assuming I got it right this time around ... Christian? #3 between 2 and 4 seems strange - I would expect once dma_buf_map_attachment() returns that the buffer can be placed in the ptes. It certianly can't be changed after the SGL is returned.. Feels like #2 should serialize all this internally? An API that returns invalidate data sometimes is dangerous :) Jason _______________________________________________ dri-devel mailing list dri-devel@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/dri-devel
