Thanks all for your responses!
At 11:27 -0800 Andrew Morton wrote:
>mb/ext3@dcs.qmul.ac.uk wrote:
>>
>> After our big ext3 file server crashes, I notice the fsck spends some time
>> replaying the journals (about 5-10 mins for all volumes on the server in
>> question). I guess it must do this should you want to mount the volumes as
>> ext2.
>
>That must be one big server. Are the fsck's running in parallel?
No. They're all volumes on an ICP-Vortex RAID 5 array, which appears as
/dev/sda, so fsck serialises it. I'm not sure to go in parallel is a good
idea, but it is hackable and if you like I'll give it a try.
It's not that big. But it does service lots of requests in parallel (it
serves a lab of about 150 PCs directly over NFS and SMB for our students,
plus various of our our servers for things like apache, tomcat and
uw-{imap,pop}d); I think it's not so much the volume of data as much as
the number of files which causes it to work hard. Since switching our
clients' Windows from NT and our NIS-GINA to Windows 2000 with roaming
profiles, the noise from the RAID box has become significantly louder :-/
At least 5-10 mins is still an order of magnitude better than the ext2
fsck + quotacheck which takes about an hour under 2.4, and about an hour
and a half under 2.2 :) Once again, thanks all for your great work.
>Oh, and how come your server crashes?
Maybe it's hardware, but we've replaced almost all the hardware since it
started crashing (but this was shortly after we switched to 2.4). I just
reckon 2.4 bruises it somewhat more than 2.2 did (NB the fsck comment
above), and its work patterns are things we can't model on any other
server. I guess if there's a race waiting to happen, it'll happen on
"custard". In fact about 90% of my bug reports here and to lkml have been
from this machine in its various guises. (The other 10% are crashes I
definitely deserved due to pathological behaviour on my part--I know I
still owe you some feedback on some of this; but the "stable" (ha ha)
servers must take precedence.) It's got all the ingredients for races:
SMP, lots of modules, many clients, multiple file serving protocols..
I'm hoping that 2.4.18 will be more stable for us. -rc1 (on another box)
is looking good.
[snip]
>I suggest that you ensure that you're getting the best possible
>parallalism in the recovery, and perhaps experiment with smaller
>journals.
Smaller journals may well be the way to go on hardware RAID systems.
Looking at (and listening to :) the array we have, I don't think it can
lose us too much throughput in our case. Next crash I'll give it a go :-/
Cheers,
Matt
