> -----Original Message----- > From: Tom Talpey [mailto:tom@xxxxxxxxxx] > Sent: Tuesday, July 21, 2015 3:47 PM > To: Steve Wise; 'Jason Gunthorpe' > Cc: 'Chuck Lever'; linux-rdma@xxxxxxxxxxxxxxx; linux-nfs@xxxxxxxxxxxxxxx > Subject: Re: [PATCH v3 05/15] xprtrdma: Remove last ib_reg_phys_mr() call site > > On 7/21/2015 7:33 AM, Steve Wise wrote: > > > > > >> -----Original Message----- > >> From: linux-rdma-owner@xxxxxxxxxxxxxxx [mailto:linux-rdma-owner@xxxxxxxxxxxxxxx] On Behalf Of Tom Talpey > >> Sent: Monday, July 20, 2015 7:15 PM > >> To: Steve Wise; 'Jason Gunthorpe' > >> Cc: 'Chuck Lever'; linux-rdma@xxxxxxxxxxxxxxx; linux-nfs@xxxxxxxxxxxxxxx > >> Subject: Re: [PATCH v3 05/15] xprtrdma: Remove last ib_reg_phys_mr() call site > >> > >> On 7/20/2015 3:41 PM, Steve Wise wrote: > >>> > >>> > >>>> -----Original Message----- > >>>> From: Tom Talpey [mailto:tom@xxxxxxxxxx] > >>>> Sent: Monday, July 20, 2015 5:04 PM > >>>> To: Steve Wise; 'Jason Gunthorpe' > >>>> Cc: 'Chuck Lever'; linux-rdma@xxxxxxxxxxxxxxx; linux-nfs@xxxxxxxxxxxxxxx > >>>> Subject: Re: [PATCH v3 05/15] xprtrdma: Remove last ib_reg_phys_mr() call site > >>>> > >>>> On 7/20/2015 2:16 PM, Steve Wise wrote: > >>>>> > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: linux-nfs-owner@xxxxxxxxxxxxxxx [mailto:linux-nfs-owner@xxxxxxxxxxxxxxx] On Behalf Of Jason Gunthorpe > >>>>>> Sent: Monday, July 20, 2015 4:06 PM > >>>>>> To: Tom Talpey; Steve Wise > >>>>>> Cc: Chuck Lever; linux-rdma@xxxxxxxxxxxxxxx; linux-nfs@xxxxxxxxxxxxxxx > >>>>>> Subject: Re: [PATCH v3 05/15] xprtrdma: Remove last ib_reg_phys_mr() call site > >>>>>> > >>>>>> On Mon, Jul 20, 2015 at 01:34:16PM -0700, Tom Talpey wrote: > >>>>>>> On 7/20/2015 12:03 PM, Chuck Lever wrote: > >>>>>>>> All HCA providers have an ib_get_dma_mr() verb. Thus > >>>>>>>> rpcrdma_ia_open() will either grab the device's local_dma_key if one > >>>>>>>> is available, or it will call ib_get_dma_mr() which is a 100% > >>>>>>>> guaranteed fallback. > >>>>>>> > >>>>>>> I recall that in the past, some providers did not support mapping > >>>>>>> all of the machine's potential physical memory with a single dma_mr. > >>>>>>> If an rnic did/does not support 44-ish bits of length per region, > >>>>>>> for example. > >>>>>> > >>>>>> Looks like you are right, but the standard in kernel is to require > >>>>>> ib_get_dma_mr, if the HCA can't do that, then it cannot be used on a > >>>>>> big memory machine with kernel ULPs. > >>>>>> > >>>>>> Looking deeper, both amso1100 and cxgb3 seem limited to 32 bits of > >>>>>> physical memory, and silently break all kernel ULPs if they are used > >>>>>> on a modern machine with > 4G. > >>>>>> > >>>>>> Is that right Steve? > >>>>>> > >>>>> > >>>>> Yes. > >>>>> > >>>>>> Based on that, should we remove the cxgb3 driver as well? Or at least > >>>>>> can you fix it up to at least fail get_dma_mr if there is too much > >>>>>> ram? > >>>>>> > >>>>> > >>>>> I would like to keep cxgb3 around. I can add code to fail if the memory is > 32b. Do you know how I get the amount of > >>> available > >>>>> ram? > >>>> > >>>> A) are you sure it's an unsigned length, i.e. is it really 31 bits? > >>>> > >>> > >>> yes. > >>> > >>>> B) why bother to check? Are machines with <4GB interesting, and worth > >>>> supporting a special optimization? > >>> > >>> No, but cxgb3 is still interesting to user applications, and perhaps NFSRDMA using FRMRs. > >> > >> I'm obviously not making myself clear. I am suggesting that cxgb3 fail > >> the ib_get_dma_mr() verb, regardless of installed memory. > >> > >> I am not suggesting it fail to load, or fail other memreg requests. It > >> should work normally in all other respects. > > > > Even with its limitation, doesn't it have utility for someone using cxgb3 in an embedded 32b environment? > > Sure, do you mean making it conditional on #if sizeof(physaddr) <= 32? > That would make sense I guess. No, a runtime check. x64 platforms will work too if the mem size takes <= 32b to describe. -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
