At 03:18 PM 10/1/2008, Talpey, Thomas wrote: >At 02:26 PM 10/1/2008, J. Bruce Fields wrote: >>On Tue, Sep 30, 2008 at 03:04:26PM -0400, Talpey, Thomas wrote: >>> At 02:44 PM 9/30/2008, J. Bruce Fields wrote: >>> >On Mon, Sep 29, 2008 at 10:07:25PM -0500, Tom Tucker wrote: >>> >> What's your perspective on the lifetime of bounce buffers, memory >>> >> windows, and the other strategies in client? >>> > >>> >I'm ignorant. Pointer to something else I should read? >>> > >>> >I assume there are similar issues on the client? >>> >>> Tom's asking about the client memory registration options and >>> whether they should all remain in the client code going forward. >>> There are different strategies in there, depending on what the >>> hardware supports, how well it supports it, and how you want >>> to run it all. >> >>Do any of them allow reading or writing memory other than that holding >>rpc data? BTW, there's a hopefully helpful slide on page 20 of the following presentation which I gave back in April to the RDMA developers: <http://openfabrics.org/archives/spring2008sonoma/Tuesday/NFS-RDMA-OFA-08.pdf> FRMR's are, in part, a result of that request. Certainly they fulfill it! Tom. > >Some of them, and the memory exposure varies. The only modes which >expose additional memory are FMR's, which expose data on the same >page as RPC data, and ALLPHYSICAL, which, well, exposes everything. > >The BOUNCEBUFFERS and REGISTER modes protect memory fiercely, >but are relatively expensive. MEMWINDOWS are safe but not very fast, >and only supported by obsolete devices. FRMR's are both safe and fast > but a) still have a cost and b) are only supported by one device at present. > >Long term, we all hope FRMRs will be the default, logical, and only choice. > >I still owe feedback on Tom's new text... it's coming soon. > >Tom. > >> >>--b. >> >>> >>> The multiple strategies stem from the early days when no two >>> adapters did quite the same thing. That said, at least one of >>> them is no longer useful - no adapters support windows today, >>> since the demise of Ammasso, although the kernel API to drive >>> them is still there in the OFA stack below. >>> >>> I think it's possible that some of the other modes can collapse, but >>> not just now. There's a lot of older hardware out there, and newer >>> hardware may appear to use the old stuff too. >>> >>> Another concern I have, frankly, is interoperability. If we collapse >>> the modes, then the temptation to assume both ends have such- >>> and-such a capability increase. If one side tries some RDMA operation >>> that is only supported properly by a certain adapter, it's hard to detect >>> that in a monoculture. Having the option to switch modes can help avert >>> this, by testing for it. >>> >>> In any case, if the current code doesn't work, it's a bug. Certainly >>> bouncebuffers (non-RDMA mode) should work perfectly and I plan to >>> check it asap. > >-- >To unsubscribe from this list: send the line "unsubscribe linux-nfs" in >the body of a message to majordomo@xxxxxxxxxxxxxxx >More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
