Hi all,
I plan on having about 100M files totaling about 8.5TiBytes. To see
how ext3 would perform with large numbers of files I've written a test
program which creates a configurable number of files into a configurable
number of directories, reads from those files, lists them and then
deletes them. Even up to 1M files ext3 seems to perform well and scale
linearly; the time to execute the program on 1M files is about double
the time it takes it to execute on .5M files. But past 1M files it
seems to have n^2 scalability. Test details appear below.
Looking at the various options for ext3 nothing jumps out as the obvious
one to use to improve performance.
Any recommendations?
Thanks!
Sean
------
Parameter one is number of files, parameter two is number of directories
to write into.
Dell MD3000 + LVM2. 2x7 10k rpm SAS disks 128k stripe RAID-0 for 3.8
TiBytes of total storage. Fedora Core 6 x86_64. 2xQuad Core Xeon.
Default mount and ext3 options used.
[root@galaxy filestore]# time /soc/abyss/test/fileSystemTest.pl 10000 1000
real 0m1.054s
user 0m0.128s
sys 0m0.382s
[root@galaxy filestore]# time /soc/abyss/test/fileSystemTest.pl 1000000 1000
real 1m0.938s
user 0m12.203s
sys 0m40.358s
[root@galaxy filestore]# time /soc/abyss/test/fileSystemTest.pl 10000000
1000
real 13m39.881s
user 2m6.645s
sys 7m26.665s
[root@galaxy filestore]# time /soc/abyss/test/fileSystemTest.pl 20000000
1000
real 44m46.359s
user 4m22.911s
sys 17m2.792s
_______________________________________________
Ext3-users mailing list
Ext3-users@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/ext3-users
