Hi Nick, > Several people have been interested to test my vfs patches, so rather > than resend patches I have uploaded a rollup against Linus's current > head. > > ftp://ftp.kernel.org/pub/linux/kernel/people/npiggin/patches/fs-scale/ > > I have used ext2,ext3,autofs4,nfs as well as in-memory filesystems > OK (although this doesn't mean there are no bugs!). Otherwise, if your > filesystem compiles, then there is a reasonable chance of it working, > or ask me and I can try updating it for the new locking. > > I would be interested in seeing any numbers people might come up with, > including single-threaded performance. Thanks for doing a rollup patch, it made it easy to test. I gave it a spin on a 64 core (128 thread) POWER5+ box. I started simple by looking at open/close performance, eg: void testcase(void) { char tmpfile[] = "/tmp/testfile.XXXXXX"; mkstemp(tmpfile); while (1) { int fd = open(tmpfile, O_RDWR); close(fd); } } At first the results were 10x slower. I took a look and it appears the MNT_MOUNTED flag is getting cleared by a remount (I'm testing on the root filesystem). This fixed it: --- fs/namespace.c~ 2009-10-15 04:34:02.000000000 -0500 +++ fs/namespace.c 2009-10-15 04:35:00.000000000 -0500 @@ -1711,7 +1711,8 @@ static int do_remount(struct path *path, else err = do_remount_sb(sb, flags, data, 0); if (!err) - path->mnt->mnt_flags = mnt_flags; + path->mnt->mnt_flags = mnt_flags | + (path->mnt->mnt_flags & MNT_MOUNTED); up_write(&sb->s_umount); if (!err) { security_sb_post_remount(path->mnt, flags, data); Attached is a before and after graph. Single thread performance is 20% faster, and we go from hitting a wall at 2 cores to scaling all the way to 64 cores. Nice work!!! Anton
Attachment:
open_close_performance.png
Description: PNG image
