On Wed, 21 Apr 2010, Chuck Lever wrote:
> So, VMware is probably not waiting for disk writes to complete, since NFS
> servers in VMware guests run faster than native. Note to self: never put
> mission critical data on NFS servers running as VMware guests.
yeah I only did this for testing...
> Still, the Linux NFS server has some kind of problem here when comparing
> apples to apples. Does increasing the number of nfsd threads on the server
> help?
Increasing the number of nfsd threads did not help (I raised them from 8 to
32 but Fedora is still crawling when extracting the tgz file, nfsstat numbers
below are with 8 nfsd threads on both Fedora and FreeBSD)
So speaking of nfsstat, I've got some new statistics, this time I extracted
the whole tgz and not just a single directory (+ files) inside it (statistics
zeroed before ofc, both tests run on bare metal and on same hardware,
network, etc.)
Fedora 12:
Server rpc stats:
calls badcalls badauth badclnt xdrcall
45960 0 0 0 0
Server nfs v3:
null getattr setattr lookup access readlink
1 0% 641 1% 6885 14% 10373 22% 18336 39% 0 0%
read write create mkdir symlink mknod
1917 4% 2552 5% 1999 4% 228 0% 0 0% 0 0%
remove rmdir rename link readdir readdirplus
633 1% 0 0% 3 0% 0 0% 0 0% 3 0%
fsstat fsinfo pathconf commit
21 0% 1 0% 1 0% 2366 5%
FreeBSD 8:
Server Info:
Getattr Setattr Lookup Readlink Read Write Create Remove
628 6421 5287 3 703 2490 1991 629
Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access
3 0 0 229 2 0 2 6526
Mknod Fsstat Fsinfo PathConf Commit
0 5 1 1 2402
Server Ret-Failed
5267
Server Faults
0
Server Cache Stats:
Inprog Idem Non-idem Misses
0 0 0 0
Server Write Gathering:
WriteOps WriteRPC Opsaved
2490 2490 0
First thing that catches my eyes is Access, it's 3x higher on Fedora, next
would be Lookup and Read, both 2x higher
MacOS 10.6 nfsstat client statistics after extracting on Fedora 12:
Client Info:
RPC Counts:
Getattr Setattr Lookup Readlink Read Write Create Remove
640 6885 10380 0 1919 2553 1999 634
Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access
3 0 0 228 1 0 3 18353
Mknod Fsstat Fsinfo PathConf Commit
0 21 1 1 2367
RPC Info:
TimedOut Invalid X Replies Retries Requests
0 0 0 0 45989
Cache Info:
Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses
84529 5034 9021 9408 5235 1701 766 2553
BioRLHits Misses BioD Hits Misses DirE Hits Misses
0 0 1 3 4 0
... after extracting on FreeBSD 8:
Client Info:
RPC Counts:
Getattr Setattr Lookup Readlink Read Write Create Remove
628 6421 5290 3 704 2490 1991 630
Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access
3 0 0 229 3 0 2 6538
Mknod Fsstat Fsinfo PathConf Commit
0 5 1 1 2402
RPC Info:
TimedOut Invalid X Replies Retries Requests
0 0 0 0 27342
Cache Info:
Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses
36508 1529 1817 4317 958 704 767 2490
BioRLHits Misses BioD Hits Misses DirE Hits Misses
1 3 1 7 3 0
Well this mostly matches what we've seen in the nfs server statistics,
even though a lot more caching was done when using the Fedora nfs server
nfsstat manpage on OSX explains them as:
Attr Hits/Misses - Performance of the NFS file attribute cache.
Lkup Hits/Misses - Performance of the directory name lookup cache.
BioR Hits/Misses - Performance of block cache for reads.
BioW Hits/Misses - Performance of block cache for writes.
BioRL Hits/Misses - Performance of symbolic link cache.
BioD Hits/Misses - Performance of directory cache.
DirE Hits/Misses - Performance of directory offset cache.
I've also captured the traffic from both sessions with tcpdump, it's quite
huge though, 42MB for Fedora and 10MB for FreeBSD... let me know if you want
to take a look
TIA
PS: Please let me say again that extraction time using FreeBSD nfs is 10
*seconds* while on Fedora it's over 5 *minutes*...
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
