Re: Poor Performance WhenNumber of Files > 1M

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



dumpe2fs reports that dir_index option is enabled. But thank you for the suggestion.

Sean

Bruno Wolff III wrote:
On Wed, Aug 01, 2007 at 18:55:53 -0700,
  Sean McCauliff <smccauliff@xxxxxxxxxxxxxxxxx> wrote:
Hi all,

I plan on having about 100M files totaling about 8.5TiBytes. To see how ext3 would perform with large numbers of files I've written a test program which creates a configurable number of files into a configurable number of directories, reads from those files, lists them and then deletes them. Even up to 1M files ext3 seems to perform well and scale linearly; the time to execute the program on 1M files is about double the time it takes it to execute on .5M files. But past 1M files it seems to have n^2 scalability. Test details appear below.

Looking at the various options for ext3 nothing jumps out as the obvious one to use to improve performance.

Any recommendations?

Did you make sure directory indexing is available? I think that is the
default now for ext3, but maybe it wasn't turned on for your test.


_______________________________________________
Ext3-users mailing list
Ext3-users@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/ext3-users

[Index of Archives]         [Linux RAID]     [Kernel Development]     [Red Hat Install]     [Video 4 Linux]     [Postgresql]     [Fedora]     [Gimp]     [Yosemite News]

  Powered by Linux