On Fri, Nov 30, 2018 at 10:41:48AM +0000, Syed Syed wrote: > > > > From: Weiny, Ira [mailto:ira.weiny@xxxxxxxxx] > > Sent: Friday, November 30, 2018 4:21 AM > > To: Jason Gunthorpe <jgg@xxxxxxxx>; Syed Syed <SYEDS@xxxxxxxxxx> > > Cc: linux-rdma@xxxxxxxxxxxxxxx > > Subject: RE: Local dma key and reserved l_key usage in mad > > > > > > > > On Thu, Nov 29, 2018 at 11:40:32AM +0000, Syed Syed wrote: > > > > Hi > > > > Just wanted to cross-check with the list on these. > > > > Is it always the case that the work requests presented on QP1(mad) > > > > shall have only "physical address" owing to the fact that the l_key > > > > associated is reserved l_key(or local_dma_key)(from 10.6.4.3.2 of the > > spec)? > > > > But is it mandatory that the consumer shall present memory from > > reserved > > > l_key on QP1? > > > > Is there a use case where normally registered memory(l_key) (and > > > > associated virtual address instead of physical) can be used in QP1 work > > > request? > > > > > > As far as I know QP0&1 is no different from any other QP and can use any > > > appropriate lkey. > > > > Agreed. > Ok. No, I was confused, since within kernel, in mad layer the wr is > comprised of dma addr (in sg_list), whereas user space verbs present > memory with virtual address. Still if someone can confirm, is > local_dma_key conceptually referring to reserved l_key (10.6.4.3.2) > of the spec? Also that reserved l_key always assumed to be referring > to physical address. Linux doesn't entirely follow the spec.. For a long time you had to call get_dma_mr() to get an all-memory physical DMA-able lkey attached to the PD. Newer chips are supposed to support LOCAL_DMA_LKEY which should follow 10.6.4.3.2, but it isn't widely implemented for some reason. Jason
