Are there any known NFS4.2 over TCP bandwidth limits ?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi

I have a NFS server with 6x100 Gb/s ports.

The NFS server connected via two switches to the ESXi hosts.  3 NFS
server ports per switch.
Many ESXi hosts connected to the same switches using 25 Gb/s ports
(4x25 Gb/s ports per single ESXi)

The NFS server exports single mount point (raid6, many NVMe SSDs PCIe Gen5)

VMs running with Oracle Linux 8.10 (kernel 4.18.0-553)

There is a "magic" bandwidth number limited up to 100Gb/s between the
NFS server and multiple VMs/ESXis.

We tried different numbers of VMs/ESXis, different fio parameters (bs,
iodepth, numjobs, reads/writes, etc...), and tried nconnect=16 option
with mount command.

Total bandwidth between the NFS server and multiple VMs/ESXis limited
up to 100 Gb/s.  Always.  While iperf between the NFS and multiple
VMs/ESXis shows hundreds Gb/s .

Anton




[Index of Archives]     [Linux Filesystem Development]     [Linux USB Development]     [Linux Media Development]     [Video for Linux]     [Linux NILFS]     [Linux Audio Users]     [Yosemite Info]     [Linux SCSI]

  Powered by Linux