David Howells <dhowells@xxxxxxxxxx> wrote:
> So, I can apply the attached patch. It opens a file each time it wants to
> write a page and then closes it again, but it's the best way to avoid the
> ENFILE problem. I'm benchmarking it at the moment.
Okay. I'm not sure why, but using ->write() in that patch appears to be _much_
better than using ->write_one_page(), despite the latter doing much less work.
The benchmark was:
mke2fs -j -O dir_index -b 4096 /dev/sda6
tune2fs -o user_xattr /dev/sda6
setenforce 0
mount /dev/sda6 /var/fscache/ -o user_xattr
insmod /tmp/sunrpc.ko
insmod /tmp/lockd.ko
insmod /tmp/nfs_acl.ko
insmod /tmp/auth_rpcgss.ko
insmod /tmp/fscache.ko
insmod /tmp/nfs.ko
insmod /tmp/cachefiles.ko
sync
service cachefilesd start
sync
mount warthog:/warthog /warthog -o fsc -t nfs4
date;
tar cf - /warthog/aaa >/dev/zero &
tar cf - /warthog/aaa2 >/dev/zero &
tar cf - /warthog/aaa3 >/dev/zero &
tar cf - /warthog/aaa >/dev/zero &
tar cf - /warthog/aaa2 >/dev/zero &
tar cf - /warthog/aaa3 >/dev/zero &
tar cf - /warthog/aaa >/dev/zero &
tar cf - /warthog/aaa2 >/dev/zero;
wait;
date
sync
echo b >/proc/sysrq-trigger
Where each of /warthog/aaa{,2,3} is about 340MB of kernel tree. The test box
only has 1GB of RAM. The server has the whole data set in RAM and is
connected by GigE with more or less no other traffic on the line.
Runs without and with the patch were alternated, and the following results
were produced:
Using write_one_page():
Thu Apr 2 22:56:45 BST 2009
Thu Apr 2 23:01:03 BST 2009 4m 18s
Thu Apr 2 23:11:12 BST 2009
Thu Apr 2 23:15:12 BST 2009 4m 0s
Thu Apr 2 23:28:39 BST 2009
Thu Apr 2 23:32:01 BST 2009 3m 22s
Using write():
Thu Apr 2 23:05:19 BST 2009
Thu Apr 2 23:07:31 BST 2009 2m 12s
Thu Apr 2 23:18:28 BST 2009
Thu Apr 2 23:20:18 BST 2009 1m 50s
Thu Apr 2 23:22:52 BST 2009
Thu Apr 2 23:25:24 BST 2009 2m 32s
So the patch improves performance by about 2x, it would appear. The disk
holding the backing fs was much more continually on with the patch, so I think
write() must be promoting some disk I/O.
Or maybe, fput() calling release() is the difference. Any thoughts?
David
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
