On Fri, 11 Jul 2008, Theodore Tso wrote:
On Fri, Jul 11, 2008 at 09:57:47AM +0000, Holger Kiehl wrote:
Thanks. Reverting that patch also fixed it for me. I was able to do my
test however performance is down another 10% (compared to
ext4-patch-queue-52c8a02a8a7b7e5915b9301e9c171b4faf22b928). ext4 is getting
slower and slower :(
How reproducible are your results? That is, if you run the benchmarks
multiple times, how much variance is there between different runs?
I always run the benchmark 3 times and then take the average. Additionally
there are two main types of test one where files are moved locally (FILE)
and another where files are send via FTP. So its 6 runs with the following
results:
FTP FILE
2548.81 6569.68
2613.05 6480.86
2599.09 6573.62
I then took all six runs added them and diveded by six giving 4564.19 fps.
Those results where achived with a5d48915 and e2fsprogs-1.41-WIP-0707.tar.bz2.
The same was done with 52c8a02a only here I used the April 27th e2fsprogs.
There I got 5054.86 fps.
Each run takes 30 minutes and approx. 10 minutes to delete the test files
from a previous run and setup the new test files, that is approx. 4 hours
for all 6 runs. I then also always do a 2 hour test run with a lot more
files and process sending files, one for FTP and one for FILE. But I
did not mention those numbers because I always did it once. But here too
one could see the numbers going down by approx. 10%.
If you are willing, this would be helpful. In your ext4 patch
repository, try out commit 179a876b. (You can do this via
"git checkout -b rc9-rebase 179a876b"; after doing the test you can
switch the working directory of the ext4 patch queue back to the master
branch via "git checkout master".) This commit is pretty much
identical to your previous 52c8a02a test, modulo rebasing to -rc9.
That is why I did another test run with ext3 which I did not mention, sorry.
Here the results:
ext3 ext4-patch-queue
52c8a02a 5536.79 5054.86
a5d48915 5587.78 4564.19
So the result of ext3 are the same while ext4-patch-queue dropped the
nearly 10%.
If you see the same results, you could try going to the next patch,
via "git checkout -b i-blocks-stat ef019f0a" which also has the fix so
that stat returns a valid i_blocks field for files that have been
freshly written when delayed allocation is enabled. Both of these
revisions rae before the patches that were causing corrupion were
added to the patch queue, so it should be fine.
The funny thing is looking at the various recent patches, I don't see
how they could be affecting performance of your patches so
significantly. I gather afdbench is very metadata intensive, with
lots of small files, but even so, none of these patches should make
that kind of difference. So that's why I'm wondering how much
variance there is between runs of afdbench, and whether that might be
a possible explanation.
Also the group descriptors still get corrupted.
Hmm, can you send me the output of dumpe2fs before and after the
benchmark run which corrupts the group descriptors? And can you send
me the output of "e2fsck -fy /dev/XXXXX >& /tmp/log", so I can see
what got corrupted?
I also note that you are using a fairly old e2fsprogs from April 27th.
You might want to try going to the just-released e2fsprogs 1.41.0
released yesterday, as that has some flex_bg layout changes that might
help out performance for afdbench.
Where those changes already in e2fsprogs-1.41-WIP-0707.tar.bz2 release?
Also note that with both the April
27th and the latest e2fsprogs 1.41.0 release, there is a mke2fs.conf
file in misc/mke2fs.conf that should be installed in /etc/mke2fs.conf
for best results.
Since I made my own src rpm, I did use the misc/mke2fs.conf in both cases.
Just checked this.
Holger
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
