Re: ext3 2.4.21 htree tests

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Stephen,

>  if you try with _1_ directory, you should see the best benefit from
> htree.

^^^
Ok, I've done that, results (1 million files in 1 directory):

create : 934 seconds (directory entry size was 22999040 bytes at end)
remove : 1315 seconds

I started to run the test on a non htree'd device and gave up as after it
had done 36000 files as it was taking 20 seconds or more to do a thousand
files (i.e. it would have taken hours to complete) (which is what I've seen
before and worked around using multiple directories).

What I did realize about my test was the way the writing was done,
essentially the directory and file is selected by doing
(file_count/100)%100, so in this case 100 files are written into the first
directory, then 100 into the next etc etc. Using a 1000 as the modulo you
get 1000 files per directory written all at once.

When re-running using 100 directories and writing the full 10000 files then
the times to create and remove are the same sort of figures as using the
1000 directories (so the constant changing of directories to write into is
nailing it) - in fact using 10 directories only takes 265 seconds to create
the files.

So htree is a **massive** boost but it does still pay to split the files
across a few directories (ok 1000 may be unncessary).

Thanks again,

Glen


_______________________________________________

Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

[Index of Archives]         [Linux RAID]     [Kernel Development]     [Red Hat Install]     [Video 4 Linux]     [Postgresql]     [Fedora]     [Gimp]     [Yosemite News]

  Powered by Linux