> -----Original Message----- > From: linux-rdma-owner@xxxxxxxxxxxxxxx [mailto:linux-rdma-owner@xxxxxxxxxxxxxxx] On Behalf Of Tom Talpey > Sent: Monday, July 20, 2015 7:15 PM > To: Steve Wise; 'Jason Gunthorpe' > Cc: 'Chuck Lever'; linux-rdma@xxxxxxxxxxxxxxx; linux-nfs@xxxxxxxxxxxxxxx > Subject: Re: [PATCH v3 05/15] xprtrdma: Remove last ib_reg_phys_mr() call site > > On 7/20/2015 3:41 PM, Steve Wise wrote: > > > > > >> -----Original Message----- > >> From: Tom Talpey [mailto:tom@xxxxxxxxxx] > >> Sent: Monday, July 20, 2015 5:04 PM > >> To: Steve Wise; 'Jason Gunthorpe' > >> Cc: 'Chuck Lever'; linux-rdma@xxxxxxxxxxxxxxx; linux-nfs@xxxxxxxxxxxxxxx > >> Subject: Re: [PATCH v3 05/15] xprtrdma: Remove last ib_reg_phys_mr() call site > >> > >> On 7/20/2015 2:16 PM, Steve Wise wrote: > >>> > >>> > >>>> -----Original Message----- > >>>> From: linux-nfs-owner@xxxxxxxxxxxxxxx [mailto:linux-nfs-owner@xxxxxxxxxxxxxxx] On Behalf Of Jason Gunthorpe > >>>> Sent: Monday, July 20, 2015 4:06 PM > >>>> To: Tom Talpey; Steve Wise > >>>> Cc: Chuck Lever; linux-rdma@xxxxxxxxxxxxxxx; linux-nfs@xxxxxxxxxxxxxxx > >>>> Subject: Re: [PATCH v3 05/15] xprtrdma: Remove last ib_reg_phys_mr() call site > >>>> > >>>> On Mon, Jul 20, 2015 at 01:34:16PM -0700, Tom Talpey wrote: > >>>>> On 7/20/2015 12:03 PM, Chuck Lever wrote: > >>>>>> All HCA providers have an ib_get_dma_mr() verb. Thus > >>>>>> rpcrdma_ia_open() will either grab the device's local_dma_key if one > >>>>>> is available, or it will call ib_get_dma_mr() which is a 100% > >>>>>> guaranteed fallback. > >>>>> > >>>>> I recall that in the past, some providers did not support mapping > >>>>> all of the machine's potential physical memory with a single dma_mr. > >>>>> If an rnic did/does not support 44-ish bits of length per region, > >>>>> for example. > >>>> > >>>> Looks like you are right, but the standard in kernel is to require > >>>> ib_get_dma_mr, if the HCA can't do that, then it cannot be used on a > >>>> big memory machine with kernel ULPs. > >>>> > >>>> Looking deeper, both amso1100 and cxgb3 seem limited to 32 bits of > >>>> physical memory, and silently break all kernel ULPs if they are used > >>>> on a modern machine with > 4G. > >>>> > >>>> Is that right Steve? > >>>> > >>> > >>> Yes. > >>> > >>>> Based on that, should we remove the cxgb3 driver as well? Or at least > >>>> can you fix it up to at least fail get_dma_mr if there is too much > >>>> ram? > >>>> > >>> > >>> I would like to keep cxgb3 around. I can add code to fail if the memory is > 32b. Do you know how I get the amount of > > available > >>> ram? > >> > >> A) are you sure it's an unsigned length, i.e. is it really 31 bits? > >> > > > > yes. > > > >> B) why bother to check? Are machines with <4GB interesting, and worth > >> supporting a special optimization? > > > > No, but cxgb3 is still interesting to user applications, and perhaps NFSRDMA using FRMRs. > > I'm obviously not making myself clear. I am suggesting that cxgb3 fail > the ib_get_dma_mr() verb, regardless of installed memory. > > I am not suggesting it fail to load, or fail other memreg requests. It > should work normally in all other respects. Even with its limitation, doesn't it have utility for someone using cxgb3 in an embedded 32b environment? -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
