On Tue, Jul 14, 2015 at 02:29:43PM -0600, Jason Gunthorpe wrote: > local_dma_lkey appears to be global, it works with any PD. > > ib_get_dma_mr is tied to a PD, so it cannot replace local_dma_lkey at > the struct device level. > > ib_alloc_pd is the in-kernel entry point, the UAPI calls > device->alloc_pd directly.. So how about the below patch as a starting > point? This looks perfect to me. After this we can get rid of the ib_get_dma_mr calls outside of ib_alloc_pd, and eventuall move setting up ->local_dma_lkey into the HW driver and kill of ib_get_dma_mr, IB_DEVICE_LOCAL_DMA_LKEY and device->local_dma_lkey. > > (2) now ULD can check for IB_DEVICE_LOCAL_DMA_LKEY and use local_dma_lkey > > in that case, or will have to perform a per-IO MR with LOCAL and > > REMOTE flags if not > > If we do the above all ULPs simply use pd->local_dma_lkey and we drop > correct uses of ib_get_dma_mr completely ? Yes, please, although: > > (3) remove insecure remote uses of ib_get_dma_mr from ULDs I kept this separate as the I suspect the "optimizations" using ib_get_dma_mr with remote flags will warrant a separate discussion. > >From 5a9799bf487d822daae5a8b8f3b197f1dda1db45 Mon Sep 17 00:00:00 2001 > From: Jason Gunthorpe <jgunthorpe@xxxxxxxxxxxxxxxxxxxx> > Date: Tue, 14 Jul 2015 14:27:37 -0600 > Subject: [PATCH] IB/core: Guarantee that a local_dma_lkey is available > > Every single ULP requires a local_dma_lkey to do anything with > a QP, so lets us ensure one exists for every PD created. > > If the driver can supply a global local_dma_lkey then use that, otherwise > ask the driver to create a local use all physical memory MR associated > with the new PD. > > Signed-off-by: Jason Gunthorpe <jgunthorpe@xxxxxxxxxxxxxxxxxxxx> Acked-by: Christoph Hellwig <hch@xxxxxx> (especially together with patches removing all other callsites of "ib_get_dma_mr(pd, IB_ACCESS_LOCAL_WRITE)". -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
