Greg Louis <glouis@dynamicro.on.ca> wrote: > > cd /root_of_tree > ./autogen.sh --arguments-to-configure > (such as --prefix=/usr or whatever; you don't really need args for this > test) Doh, OK, I needed to update automake & autoconf. > > That was a bit optimistic. The script Matthias was thinking of is a > shuffle filter; takes stdin and outputs the lines in random order. I > can't give you a script that creates the input you need to reproduce > the problem. However, if you can download a 4Mb file from > http://www.bgl.nu/bogofilter/randlist.txt.bz2 > and uncompress it, then you can do > mkdir scratch > ./bogoutil -l scratch/newdb <randlist.txt > and that will take ca half an hour on ext3/ordered but around 4 minutes > on ext2; a bit longer than 4 minutes on ext3/writeback. Fast drives > might shrink those times in proportion. Ah lovely, thanks. I tried this on my 800MHz/256M laptop (running 2.5.59-mm7++) and yeah, it sucks. So I did: mount /dev/hda6 -o remount,commit=120 and lo, bogoutil -l newdb < ~/randlist.txt 19.06s user 28.04s system 95% cpu 49.468 total Look at the `vmstat 1' output during the normal commit=5 run: procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu---- r b swpd free buff cache si so bi bo in cs us sy id wa 1 2 0 4544 26476 177576 0 0 4 11 12 11 0 0 99 1 0 2 0 4544 26476 177576 0 0 0 1080 1246 96 0 1 0 99 0 2 0 5384 26460 176552 0 0 4 784 1172 81 0 2 0 98 0 2 0 5392 26460 176552 0 0 0 880 1197 70 0 1 0 99 0 2 0 5392 26460 176552 0 0 0 876 1204 79 0 0 0 100 0 2 0 5392 26460 176552 0 0 0 916 1211 80 0 1 0 99 0 2 0 5392 26460 176556 0 0 0 792 1182 60 0 1 0 99 0 2 0 5392 26460 176568 0 0 0 840 1187 84 1 1 0 98 0 2 0 5392 26460 176572 0 0 0 992 1216 75 1 2 0 97 Very dribbly, and it's doing a huge amount of I/O. vmstat 1 with commit=120: 1 0 0 5920 26576 176260 0 0 0 0 1004 9 38 62 0 0 1 0 0 5500 26580 176664 0 0 0 0 1004 11 38 62 0 0 1 0 0 4980 26580 177176 0 0 0 0 1004 9 38 62 0 0 1 0 0 4568 26580 177584 0 0 0 0 1003 9 38 62 0 0 1 0 0 5280 26552 176868 0 0 0 0 1004 11 39 61 0 0 1 0 0 4832 26552 177324 0 0 0 0 1005 9 40 60 0 0 1 0 0 4440 26552 177696 0 0 0 0 1004 11 39 61 0 0 1 0 0 5112 26524 177060 0 0 0 0 1004 11 40 60 0 0 1 0 0 4608 26524 177552 0 0 0 0 1004 9 41 59 0 0 1 0 0 5224 26496 176944 0 0 0 0 1004 11 39 61 0 0 1 0 0 4776 26500 177380 0 0 0 0 1004 9 39 61 0 0 1 0 0 5392 26444 176868 0 0 0 0 1004 13 36 64 0 0 1 0 0 4832 26444 177428 0 0 0 0 1004 9 39 61 0 0 1 0 0 5448 26396 176796 0 0 0 0 1004 11 38 62 0 0 1 0 0 4888 26396 177348 0 0 0 0 1004 9 36 64 0 0 1 0 0 4496 26396 177748 0 0 0 0 1003 9 38 62 0 0 0 1 0 5616 25704 176948 0 0 0 15188 1003 14 34 56 0 10 0 1 0 5616 25704 176948 0 0 0 7696 1075 27 0 2 0 98 0 1 0 5740 25704 176948 0 0 0 520 1106 50 0 1 0 99 0 0 0 6400 25712 176948 0 0 0 52 1037 40 0 2 74 24 0 0 0 6400 25712 176948 0 0 0 0 1004 9 0 0 100 0 blam. Now, looking at the enormous amount of system time which the commit=120 run took, I assume that the application is doing a _ton_ of overwriting. Redirtying the same pages again and again and again. So poor old ext3 keeps rewriting them again and again. You'll hit similar problems with ext2 - on a slower computer, or on a larger database, or on a system with the kupdate interval decreased from the 30 second default. My recommendation is to work out why the application is performing so much overwriting, and make it stop. Now without a kernel profile that's all speculation. I need to reboot this machine to run a profile, so I'm going to hit send on this lot ;) _______________________________________________ Ext3-users@redhat.com https://listman.redhat.com/mailman/listinfo/ext3-users
