On 04/20/2010 05:21 PM, Stefan Krüger wrote:
On Mon, 19 Apr 2010, Chuck Lever wrote:
On 04/19/2010 08:21 AM, Stefan Krüger wrote:
On Thu, 15 Apr 2010, Stefan Krüger wrote:
Hello list,
I have some really strange nfs performance issues
NFS server is Fedora 12, running
* kernel-2.6.32.11-99.fc12.x86_64 and
* nfs-utils-1.2.1-4.fc12.x86_64
* nfs shared /home is ext4 with default mount options
/etc/exports:
/home 192.168.1.0/255.255.255.0(rw,sync)
nfs and nfslock are up and running
Nothing else touched on the server nfs-wise.
NFS client is Mac OS X, version 10.6.3
My /home dir is automounted on the Mac with the following mount options:
* nosuid,nodev,resvport,rdirplus,rwsize=1048576
(nfsv3 and tcp are default, I have also tried udp, and with and without
rdirplus, with different read/write sizes (started with 32k, less for udp,
and then cranked it up to 1m to make the beachball appear less often),
but I still have issues no matter which options I chose)
Anyway, I'm stuck now, surfing the web with Safari is a very unpleasant
experience on nfs, beachball every now and then together with a huge amount
of network traffic (RX with 20MB/s+ peaks), not unusual to see several
gigabytes received after some minutes browsing, XCode shows a ''The
document "SomeFile.m" could not be saved.''-error after some edits, Opera
hangs for minutes when closing, etc etc.
It's horrible :(
Another example, extracting
http://www.bignerdranch.com/solutions/Cocoa-3rd.tgz took over 3min!
$ time tar xzf Cocoa-3rd.tgz
0.169u 3.198s 5:51.10 0.9% 0+0k 1+6972io 0pf+0w
$ time rm -rf Solutions-Cocoa-3rd/
0.014u 0.477s 0:45.59 1.0% 0+0k 1+1io 0pf+0w
So any help or hints really appreciated
So, no answers yet, but I did some more tests, i.e. I tried extracting the
Cocoa-3rd.tgz (2.2MB, 12MB untar'ed) on FreeBSD 8.0-REL (running inside
VMWare though), and still it was much faster (5:51.10 vs 0:09.35) than
extracting on bare metal fedora12:
$ time tar xfz Cocoa-3rd.tgz
0.104u 1.474s 0:09.35 16.7% 0+0k 0+4896io 0pf+0w
$ time rm -rf Solutions-Cocoa-3rd
0.006u 0.160s 0:01.24 12.9% 0+0k 0+0io 0pf+0w
I captured the nfs traffic with tcpdump (tcpdump -i eth1 -s 0 -w nfs.out
host nfssrv and port 2049) on both freebsd8 (interface for freebsd is a bit
different ofc) and fedora12 while running
tar xfz Cocoa-3rd.tgz Solutions-Cocoa-3rd/02_GetStarted
(which extracts just a couple of files) , you can find them here:
Fedora 12 tcpdump -> http://www.dpaste.org/5cvp/
FreeBSD 8 tcpdump -> http://www.dpaste.org/uCGX/
The number of packets is around 1800 for the FreeBSD server and around
1940 for the Linux server. The RPC counts you posted in a later email
show that Linux does more LOOKUP and ACCESS requests. But generally, it
looks like your client is doing roughly the same amount of work in both
cases.
But what catches my eye in the F12 tcpdump is that there are pauses
where the server reply is delayed by a few milliseconds after a SETATTR
or COMMIT. This looks normal, since disk writes can take a few
milliseconds.
FreeBSD doesn't appear to have these pauses, so I suspect FreeBSD is
doing something illegal. No NFS server can turn a SETATTR around in
just a few microseconds and claim that it is on permanent storage,
unless it has some kind of NVRAM.
First of all, thanks for your answer Chuck :)
there are some additional packets because the .tgz was on the same
(nfs-mounted) dir and on a local dir during the freebsd test (so some extra
reads etc. sneaked in the linux tcpdump/nfsstat -s)
Just for the record, Solaris 10u8 (UFS) extraction time is almost the same as
FreeBSDs:
$ time tar xfz Cocoa-3rd.tgz
0.111u 1.621s 0:09.03 19.1% 0+0k 8+4966io 0pf+0w
So I guess they're both cheating ;-)
Is Solaris running in a guest too?
What physical file system are you using on the Linux NFS server?
--
chuck[dot]lever[at]oracle[dot]com
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
