On Wed, Apr 24, 2013 at 11:26 AM, Tom Talpey <tom@xxxxxxxxxx> wrote: >> On Wed, Apr 24, 2013 at 9:27 AM, Wendy Cheng <s.wendy.cheng@xxxxxxxxx> >> wrote: >>> >> So I did a quick read on sunrpc/xprtrdma source (based on OFA 1.5.4.1 >> tar ball) ... Here is a random thought (not related to the rb tree >> comment)..... >> >> The inflight packet count seems to be controlled by >> xprt_rdma_slot_table_entries that is currently hard-coded as >> RPCRDMA_DEF_SLOT_TABLE (32) (?). I'm wondering whether it could help >> with the bandwidth number if we pump it up, say 64 instead ? Not sure >> whether FMR pool size needs to get adjusted accordingly though. > > 1) > > The client slot count is not hard-coded, it can easily be changed by > writing a value to /proc and initiating a new mount. But I doubt that > increasing the slot table will improve performance much, unless this is > a small-random-read, and spindle-limited workload. Hi Tom ! It was a shot in the dark :) .. as our test bed has not been setup yet .However, since I'll be working on (very) slow clients, increasing this buffer is still interesting (to me). I don't see where it is controlled by a /proc value (?) - but that is not a concern at this moment as /proc entry is easy to add. More questions on the server though (see below) ... > > 2) > > The observation appears to be that the bandwidth is server CPU limited. > Increasing the load offered by the client probably won't move the needle, > until that's addressed. > Could you give more hints on which part of the path is CPU limited ? Is there a known Linux-based filesystem that is reasonbly tuned for NFS-RDMA ? Any specific filesystem features would work well with NFS-RDMA ? I'm wondering when disk+FS are added into the configuration, how much advantages would NFS-RDMA get when compared with a plain TCP/IP, say IPOIB on CM , transport ? -- Wendy -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
