RE: [PATCH V3 1/5] RDMA/core: Transport-independent access flags

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




> -----Original Message-----
> From: 'Christoph Hellwig' [mailto:hch@xxxxxxxxxxxxx]
> Sent: Tuesday, July 14, 2015 2:45 PM
> To: Steve Wise
> Cc: 'Sagi Grimberg'; 'Steve Wise'; 'Jason Gunthorpe'; 'Tom Talpey'; 'Doug Ledford'; 'Christoph Hellwig'; sagig@xxxxxxxxxxxx;
> ogerlitz@xxxxxxxxxxxx; roid@xxxxxxxxxxxx; linux-rdma@xxxxxxxxxxxxxxx; eli@xxxxxxxxxxxx; target-devel@xxxxxxxxxxxxxxx; linux-
> nfs@xxxxxxxxxxxxxxx; trond.myklebust@xxxxxxxxxxxxxxx; bfields@xxxxxxxxxxxx; 'Oren Duer'
> Subject: Re: [PATCH V3 1/5] RDMA/core: Transport-independent access flags
> 
> On Tue, Jul 14, 2015 at 02:25:50PM -0500, Steve Wise wrote:
> > if (device_supports_fastreg && device_supports_signature)
> >         use FRMR
> > else
> >         use DMAMR
> >
> > Shouldn't we just recode it this way?
> >
> > if (device_supports_fastreg)
> >         use FRMR
> > else
> >         use DMAMR
> 
> How does IB_DEVICE_LOCAL_DMA_LKEY (and ->local_dma_lkey) play into
> this?  Seems like that should be the preferred option if supported.
>

The local_dma_lkey can be used in any rdma sge that requires an lkey.  It is supported for kernel-mode only.  So if you're only ever
going to use the lkey for IO, you really don't need a DMA_MR at all, unless you want to somehow partition up your work load by
protection domain.  But I claim for lkeys, the PD doesn't really protect anything since the remote peers can't use it anyway.  

> Interestingly enough various iWarp driver seem to support this option,
> what's the story behind that?  The (to me surprising) conclusion on
> the list was that iWarp would always need a memory regireations that
> also allows remove writes even for lkeys, but from looking at the
> users of IB_DEVICE_LOCAL_DMA_LKEY / local_dma_lkey that seem to
> prefer that over creating a MR.

There is confusion about lkeys and rkeys with regard to iWARP.  In the iWARP verbs, there is no distinction between an lkey and
rkey: they are the same key, called a Steering Tag or STAG.  When you create a MR, the lkey == rkey == STAG for iwarp transports.
Somewhat related, but really a different issue, is that SGEs that are the target of a read need REMOTE_WRITE access flags on their
STAG for iWARP. 

Clear as mud? :)


--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [Linux Filesystem Development]     [Linux USB Development]     [Linux Media Development]     [Video for Linux]     [Linux NILFS]     [Linux Audio Users]     [Yosemite Info]     [Linux SCSI]

  Powered by Linux