Re: Poor Performance WhenNumber of Files > 1M

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Aug 01, 2007 at 18:55:53 -0700,
  Sean McCauliff <smccauliff@xxxxxxxxxxxxxxxxx> wrote:
> Hi all,
> 
> I plan on having about 100M files totaling about 8.5TiBytes.   To see 
> how ext3 would perform with large numbers of files I've written a test 
> program which creates a configurable number of files into a configurable 
> number of directories, reads from those files, lists them and then 
> deletes them.  Even up to 1M files ext3 seems to perform well and scale 
> linearly; the time to execute the program on 1M files is about double 
> the time it takes it to execute on .5M files.  But past 1M files it 
> seems to have n^2 scalability.  Test details appear below.
> 
> Looking at the various options for ext3 nothing jumps out as the obvious 
> one to use to improve performance.
> 
> Any recommendations?

Did you make sure directory indexing is available? I think that is the
default now for ext3, but maybe it wasn't turned on for your test.

_______________________________________________
Ext3-users mailing list
Ext3-users@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/ext3-users

[Index of Archives]         [Linux RAID]     [Kernel Development]     [Red Hat Install]     [Video 4 Linux]     [Postgresql]     [Fedora]     [Gimp]     [Yosemite News]

  Powered by Linux