On Fri, Feb 27, 2009 at 08:31:01AM +0100, Andrzej Szymański wrote:> > Does anybody know how to avoid the file fragmentation when a file is > created over NFSv3?> > A file created locally is OK:> dd bs=32k if=/dev/zero of=test count=32x1024 conv=fsync> filefrag test> test: 10 extents found, perfection would be 9 extents> > When I create the file in the same dir, but from another machine, > mounted over NFS:> > filefrag test> test: 4833 extents found, perfection would be 9 extents> > With such a file a sequential read is quite slow (~76MB vs >200MB on my > raid card).> > I can just suspect that this is a problem of block allocation when the > same file is appended by different processes (8 NFS threads).> > I've tried mounting ext3 with -o reservation and switch to NFS over TCP, > with no improvement.> > Both systems are Centos 5.2 with kernel 2.6.18-92.1.22.el5> The ext3 is mounted with rw,nosuid,nodev,usrquota,grpquota,acl> NFS export: rw,sync,no_root_squash> 8 NFS threads.> Remotely mounted with options > rw,intr,nfsvers=3,proto=udp,rsize=32768,wsize=32768> > I would be very grateful for any help.> > Andrzej
First watch out for comparing sparse files and real files.
dd bs=32k if=/dev/zero of=test count=32x1024 conv=fsync
Note that dev/zero combined with dd may be building a sparse file (or not)Sparse file block allocation is very different.I would build up a large file of binary data and dd it into testhaving been bitten by sparse file filesystem tricks.
Also a local filesystem can have a very different free listthan your NFS file system's underlying FS. You need to dothe comparison on the exact same filesystem with the onlydifference being that one case is local and the other NFS.If I run your dd on my /tmp I get 18 extents while on /var/tmpI get 582 extents. Both are local to this system. So 18 localand 582 local tells me that you must test exactly the same FSwith the only difference is that the creation was local .vs. NFS.
All in all this is a don't care -- extents are not exactly equivalent todisk seeks and other disk I/O issues.
Some of this can be improved only if you rebuild the file system. mkfshas a lot of flags and choices... You might also need to switchfilesystems -- xfs, ext2, ext3, ext4, jfs, reiser...
To some extent if you make an ideal local copy of a badly fragmentedfile you can improve the layout on disk/ filesystem. This should onlybe considered for very long lived very large files. Making a copy andcomparing the original and copy with filefrag can tell you if this isworth doing. Backup and restore can help. As a filesystem gets fullthis will get worse and worse. If you are +60% full do not bother.
-- T o m M i t c h e l l Found me a new hat, now what?
_______________________________________________CentOS mailing listCentOS@xxxxxxxxxxxxxx://lists.centos.org/mailman/listinfo/centos
