Re: [PATCH v3 05/15] xprtrdma: Remove last ib_reg_phys_mr() call site

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Jul 20, 2015, at 6:41 PM, Jason Gunthorpe <jgunthorpe@xxxxxxxxxxxxxxxxxxxx> wrote:

> On Mon, Jul 20, 2015 at 06:31:11PM -0400, Chuck Lever wrote:
>> 
>> On Jul 20, 2015, at 6:26 PM, Jason Gunthorpe <jgunthorpe@xxxxxxxxxxxxxxxxxxxx> wrote:
>> 
>>> On Mon, Jul 20, 2015 at 03:03:11PM -0400, Chuck Lever wrote:
>>>> +	iov->length = size;
>>>> +	iov->lkey = ia->ri_have_dma_lkey ?
>>>> +				ia->ri_dma_lkey : ia->ri_bind_mem->lkey;
>>>> +	rb->rg_size = size;
>>>> +	rb->rg_owner = NULL;
>>>> 	return rb;
>>> 
>>> There is something odd looking about this..
>>> 
>>> ri_bind_mem is only setup in the RPCRDMA_ALLPHYSICAL and
>>> RPCRDMA_MTHCAFMR cases.
>>> 
>>> RPCRDMA_FRMR doesn't set it up.
>>> 
>>> So this code in rpcrdma_alloc_regbuf is never called for the FRMR
>>> case?
>>> 
>>> If yes, then, how is FRMR working? There is absolutely no reason to
>>> use FRMR to register local send buffers, just use the global all
>>> memory lkey...
>>> 
>>> If no, then that is an oops?
>> 
>> I’ve tested this code, no oops.
>> 
>> FRWR always uses the DMA lkey. xprtrdma does not use FRWR if
>> IB_DEVICE_LOCAL_DMA_LKEY is not asserted.
> 
> Ah, I see. Ok.
> 
> Is there a reason to link FRWR and LOCAL_DMA_LKEY together?

Tom would know.

> Just use
> the code you have for fmr with frwr to get the lkey, probably move it
> to rpcrdma_ia_open .. Physical MR should create a 2nd MR dedicated for
> rkey use.
> 
> That will work really well with the series I'm working on:
> 
> https://github.com/jgunthorpe/linux/tree/remove-ib_get_dma_mr
> 
> To just drop ib_get_dma_mr entirely.

Yes, I sorted this out so it would be obvious how to deal with
xprtrdma when the time comes. All three registration modes can
just use pd->local_dma_lkey for the SEND/RECV buffers.

PHYSICAL would still need ib_get_dma_mr() (or something equivalent)
for MRs needing remote access, agreed.

--
Chuck Lever



--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Filesystem Development]     [Linux USB Development]     [Linux Media Development]     [Video for Linux]     [Linux NILFS]     [Linux Audio Users]     [Yosemite Info]     [Linux SCSI]

  Powered by Linux