> On Apr 28, 2022, at 9:05 AM, Dennis Dalessandro <dennis.dalessandro@xxxxxxxxxxxxxxxxxxxx> wrote: > > Hi NFS folks, > > I've noticed a pretty nasty regression in our NFS capability between 5.17 and > 5.18-rc1. I've tried to bisect but not having any luck. The problem I'm seeing > is it takes 3 minutes to copy a file from NFS to the local disk. When it should > take less than half a second, which it did up through 5.17. > > It doesn't seem to be network related, but can't rule that out completely. > > I tried to bisect but the problem can be intermittent. Some runs I'll see a > problem in 3 out of 100 cycles, sometimes 0 out of 100. Sometimes I'll see it > 100 out of 100. It's not clear from your problem report whether the problem appears when it's the server running v5.18-rc or the client. It looks vaguely like a recent client issue where it encounters a memory shortage while preparing an RPC and a delay is needed. > The latest attempt I have is at 5.18-rc4 and here's what I see when I 1) create > file with dd, 2) copy local disk to NFS mount, 3) copy NFS to local disk. > > Test 2 > Creating /dev/shm/run_nfs_test.junk... > 262144+0 records in > 262144+0 records out > 268435456 bytes (268 MB, 256 MiB) copied, 1.29037 s, 208 MB/s > Done > copy /dev/shm/run_nfs_test.junk to /mnt/nfs_test/run_nfs_test.junk... > > real 0m0.502s > user 0m0.001s > sys 0m0.250s > Done > copy /mnt/nfs_test/run_nfs_test.junk to /dev/shm/run_nfs_test.tmp... > > real 4m11.835s > user 0m0.001s > sys 0m0.277s > Done > Comparing files... > Done > Remove /dev/shm/run_nfs_test.tmp... > > real 0m0.031s > user 0m0.000s > sys 0m0.031s > Done > Remove /dev/shm/run_nfs_test.junk... > > real 0m0.031s > user 0m0.001s > sys 0m0.030s > Done > Remove /mnt/nfs_test/run_nfs_test.junk... > > real 0m0.024s > user 0m0.001s > sys 0m0.022s > Done > > Create the server with a 4G RAM disk: > mount -t tmpfs -o size=4096M tmpfs /mnt/nfs_test > > exportfs -o fsid=0,rw,async,insecure,no_root_squash > 192.168.254.0/255.255.255.0:/mnt/nfs_test > > Mount on the client side with: > mount -o rdma,port=20049 192.168.254.1:/mnt/nfs_test /mnt/nfs_test > > Any advice on tracking this down further would be greatly appreciated. > > Thanks > > -Denny > -- Chuck Lever
