Hello,
I am an HP engineer participating in L3 Linux support. Recently we have found
that current design of nfs_getattr() might create huge delays in stat() on
the file we are writing to (this is important for big files only, >2Gb).
The problem
-----------
Assuming that /nfs is an NFS-mounted FS:
1. In one shell, start
$ dd if=/dev/zero of=/nfs/dir/big bs=1G count=20
2. In another shell, start
$ ls -l /nfs/dir
or
$ ls -l /nfs/dir/big
'ls' does not return until the whole /nfs/dir/big is written.
Analysis
--------
Kernel 2.6.16 has introduced the following change in nfs_getattr():
NFS: Make stat() return updated mtimes after a write()
The SuS states that a call to write() will cause mtime to be updated on
the file. In order to satisfy that requirement, we need to flush out
any cached writes in nfs_getattr().
Speed things up slightly by not committing the writes.
Signed-off-by: Trond Myklebust <Trond.Myklebust@xxxxxxxxxx>
+ /* Flush out writes to the server in order to update c/mtime */
+ nfs_sync_inode(inode, 0, 0, FLUSH_WAIT|FLUSH_NOCOMMIT);
Then later:
2.6.16-rc6:
http://www.linux-nfs.org/Linux-2.6.x/2.6.16-rc6/linux-2.6.16-99-fix_nfs_sync_inode_race.dif
/* Flush out writes to the server in order to update c/mtime */
- nfs_sync_inode(inode, 0, 0, FLUSH_WAIT|FLUSH_NOCOMMIT);
+ nfs_sync_inode_wait(inode, 0, 0, FLUSH_NOCOMMIT);
I understand the reasoning behind that. From application point of view, NFS
file/directory should behave the same as on local FS. If we have queued many
writes, without this patch stat() will return incorrect results, both for
mtime and file length. Some applications may depend on stat() results being
correct.
At the same time, the fact that we have to wait forever while copying big
files and doing 'ls -l' on that directory (or on the file being written) is
not very good either (two HP customers have complained about this after
migrating from RHEL4 to RHEL5).
The problem is still there in 2.6.27. I am not sure what can be done to both
reduce the stat() delay and guarantee reasonable stat() results.
It is interesting that with 'noac' stat() returns much faster (just 1-3s
delay).
Best regards,
Alex
--
------------------------------------------------------------------
Alexandre Sidorenko email: asid@xxxxxx
Global Solutions Engineering: Unix Networking
Hewlett-Packard (Canada)
------------------------------------------------------------------
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
