On Mon, Feb 18, 2019 at 01:46:24PM -0800, Chris Tracy wrote: > Hopefully I'm not missing something obvious, but I'm curious > whatever happened to the patch series from late 2012 that added > dynamic v4.1 session slot allocation support to nfsd: > > https://www.spinics.net/lists/linux-nfs/msg34390.html > > The corresponding nfs client patches were integrated, but the nfsd > series seems to have been left out due to release timing: > > https://www.spinics.net/lists/linux-nfs/msg34505.html > > However, they don't seem to ever have been integrated or discussed > again. Were there other issues that prevented its inclusion in the > intervening time? They'd probably need reworking. The latest discussion I can find is: https://lore.kernel.org/linux-nfs/CAABAsM6vDOaudUZYWH23oGiWGqX5Bd1YbCDnL6L=pxzMXgZzaw@xxxxxxxxxxxxxx/ > Alternatively, is there some admin-tweakable knob for controlling > the number of slots available per-session on the NFS v4.1 server > (nfsd.ko), similar to the 'max_session_slots' client-side parameter > for nfs.ko? > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/fs/nfs?id=ef159e9177cc5a09e6174796dde0b2d243ddf28b > > I ask because I'm currently standing up a (very) modest HPC cluster > PoC (1 server, 8 client nodes, all 10Gbit, all running CentOS 7.6) > and figured that was a good enough excuse to finally move away from > NFS v3 and investigate NFS v4.x. However, initial performance > testing showed that while NFS v4.0 was essentially identical to v3, > NFS v4.2 (and v4.1) were around 25% slower. > > Looking at the traffic in wireshark, I see that in CREATE_SESSION, > the client sets ca_maxrequests to 64 (consistent with the value of > 'max_session_slots') but the server always replies with a value of > 10 for ca_maxreqests. This seems to be the source of the > performance issue, since if I fallback to v4.0 or v3, but set nfsd > to use only 10 threads in nfs.conf, I get roughly equivalent > performance to v4.2. > > Looking at the code (both in CentOS's 3.10.0-957.5.1.el7.x86_64 and > in the 4.20.8 mainline), it seems the value that would need to > change is the preprocessor define NFSD_CACHE_SIZE_SLOTS_PER_SESSION. > This is fixed at 32, and while it's a bit more complex than this, > the code in nfs4_get_drc_mem (fs/nfsd/nfs4state.c) basically sets > the per-client session slot limit to '(int)(32/3)' which is where > the '10' comes from. Thanks for the report! I think the limit should only be that low if the client requests very large slots. Do your clients have 35c036ef4a72 "nfs: RPC_MAX_AUTH_SIZE is in bytes" applied? > Is there something else I've missed somewhere that allows adjusting > the server-side session slot limit to be more than 10 without having > to compile a custom version of nfsd.ko? No. It might be a good idea, though really I think your setup should just work out of the box. --b.
