NFS4.x works best with newer kernels. Maybe you could try with an elrepo
kernel.
Increase the number of NFS threads on the NFS server (default is 8,
which is too little).
|RPCNFSDCOUNT=32|
This is an example of kernel tunning for 10Gb network, adjust to your setup:
vm.min_free_kbytes = 1048576
vm.swappiness = 10
net.core.somaxconn = 4096
# set max buffers to 64MB
net.core.rmem_max = 67108864
net.core.wmem_max = 67108864
# increase Linux autotuning TCP buffer size to 32MB
net.ipv4.tcp_rmem = 4096 87380 33554432
net.ipv4.tcp_wmem = 4096 65536 33554432
net.core.netdev_max_backlog = 250000
net.ipv4.neigh.default.unres_qlen = 100
net.ipv4.neigh.ens3f0.unres_qlen = 100
net.ipv4.neigh.ens3f1.unres_qlen = 100
fs.file-max = 98584540
You can calculate the buffer size with this tool:
https://www.switch.ch/network/tools/tcp_throughput/?do+new+calculation=do+new+calculation
<https://www.switch.ch/network/tools/tcp_throughput/?do+new+calculation=do+new+calculation>
Enable readahead for the filesystem for instance:
blockdev --setra 16384 /dev/mapper/mpatha1
for x in sd[a-d]; do blockdev --setra 16384 /dev/$x ; done
It also helps if you disable flowcontrol on the network cards (and
switches).
You could also try to reduce the maximum flow rate on the NFS host (set
max rate to 8gbit for example)
It would be best to test the actual speed by iperf3 test as someone else
already suggested. But since the performance it so bad, I would suspect
that the source of the problem is in the the network, not the server.
Cheers,
Barbara
On 5/29/22 01:52, Valeri Galtsev wrote:
On 5/28/22 9:43 AM, Erik Frangež via CentOS wrote:
Hi guys,
we are setting NFS server on CentOS7 system. Everything working OK
except speed, speed over NFS very drop... if we run dd command
directly on server we are getting speed around 1,4Gbps, if we run
from client connected to NFS is 200Mbps.
Do you have maybe some advice what we need to check?
Make sure you have good speed of physical connection along the whole
path from server to client(s), enable Jumbo packets on all switches
along the path. NFS experts will add NFS specific tuning.
Valeri
Thank you!
Best, Erik
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos
