James Pearson wrote: > > We have a number of identical NFS clients mounting a server using > NFSv4.1 - server and clients are all running CentOS 7.5 (kernel > 3.10.0-862.14.4.el7.x86_64) > > However, on some clients, the NFS performance 'degrades' with time ... > > Running a simple test - a python script that just imports a module > (python and its modules are installed on the NFS share) can be an order > of magnitude or more slower on some clients. i.e. very little data is > transferred, it is the rate of stat'ing and opening files on the NFS > server that is 'slow' > > Running a tcpdump on a 'slow' client shows that the NFS traffic > generated on the 'slow' client is again an order of magnitude or more > when compared with that generated by a 'fast' client > > The majority of the extra NFS traffic in the slow case, appears to be a > large number of NFS 'TEST_STATEID' calls the client makes - which are > not there in the tcpdump on the fast client > > The issue can be 'fixed' in the short term by rebooting the affected > client - and after a reboot, running the same tcpdump shows no > TEST_STATEID calls - however after a while (several days), the > performance might degrade again > > I've found a number of reports of excessive TEST_STATEID calls - but > most seem to relate to NFSv4 client hangs - which is not happening here > - things are working, but much slower than they should be ... > > Has anyone come across this issue - and have any fixes/workarounds? After a bit of further poking about, it looks like we are hitting the issue described in: https://bugzilla.redhat.com/show_bug.cgi?id=1552203 The fix should be in 7.7, but the linked: https://access.redhat.com/solutions/3915571 (login required) has a few suggested workarounds which includes mounting with NFSv4.0 or disabling 'NFSv4 delegations' on the server via: sysctl -w fs.leases-enable=0 I've used the above sysctl which appears to have fixed the issue - although I had to restart NFS on the server to notice any change with the affected clients - so I'm not sure if the sysctl change or the NFS restart 'fixed' the issue ... James Pearson _______________________________________________ CentOS mailing list CentOS@xxxxxxxxxx https://lists.centos.org/mailman/listinfo/centos