How about the file size ? If the size is small, another performance kill should be on disk inode layout. Because the order of access dentries of dir is probably different from the order of allocating inodes in inode tables. This will make much time wast on hard disk seeking for the first time. Just FYI. Coly Sean McCauliff wrote: > Hi all, > > I plan on having about 100M files totaling about 8.5TiBytes. To see > how ext3 would perform with large numbers of files I've written a test > program which creates a configurable number of files into a configurable > number of directories, reads from those files, lists them and then > deletes them. Even up to 1M files ext3 seems to perform well and scale > linearly; the time to execute the program on 1M files is about double > the time it takes it to execute on .5M files. But past 1M files it > seems to have n^2 scalability. Test details appear below. > > Looking at the various options for ext3 nothing jumps out as the obvious > one to use to improve performance. > > Any recommendations? > > Thanks! > Sean > > ------ > Parameter one is number of files, parameter two is number of directories > to write into. > > Dell MD3000 + LVM2. 2x7 10k rpm SAS disks 128k stripe RAID-0 for 3.8 > TiBytes of total storage. Fedora Core 6 x86_64. 2xQuad Core Xeon. > Default mount and ext3 options used. > > [root@galaxy filestore]# time /soc/abyss/test/fileSystemTest.pl 10000 1000 > > real 0m1.054s > user 0m0.128s > sys 0m0.382s > > [root@galaxy filestore]# time /soc/abyss/test/fileSystemTest.pl 1000000 > 1000 > > real 1m0.938s > user 0m12.203s > sys 0m40.358s > [root@galaxy filestore]# time /soc/abyss/test/fileSystemTest.pl 10000000 > 1000 > > real 13m39.881s > user 2m6.645s > sys 7m26.665s > [root@galaxy filestore]# time /soc/abyss/test/fileSystemTest.pl 20000000 > 1000 > > real 44m46.359s > user 4m22.911s > sys 17m2.792s _______________________________________________ Ext3-users mailing list Ext3-users@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/ext3-users
