Hello Everyone, I've been enjoying this discussion on file systems. I've written a few myself, and have some concerns and annoyances with some of the older Unix file systems. From the replies, it appears we have some real experts here, so, please indulge my posting... I would like to hear commentary, and hopefully learn something about these new file system choices, and I'd like to start this discussion by making some observations to provide food for thought. To wit: My first job in Comp-Sci was to write a real-time, multi-tasking operating system and then _hand_optimize_ it to the machine cycle. Included in this work was to write the disk driver and file system. And, the hardware had a bank-switchable memory cache, which I incorporated into the I/O system. This gave me a real appreciation for file systems and performance. One observation I have is that many people have a bizarre, one-sided view of performance. This is true of a vast number of human activities, such as the automobile driver who focuses on driving on dry, smooth pavement and forgets s/he drives in the wet on rough roads. ...And, it's common enough throughout the disciplines of computer science, such as the database engineer who focuses on transactions per second and forgets about failure recovery time... And so it is with file systems: Some aspects of performance which I find are critical of a file system seem to be completely ignored or at least discounted and put as low priority by current practitioners. (...I have left below the previous keystrokes of this thread for easy review to see what I'm talking about.) What are these over-looked aspects of performance? Two things: Data security (safty) and system restart times. Here's my imperative: Every change to the structure on disk _must_be_ written to disk that very instant. On-disk structure changes are _the_ most critical aspect. Caching disk structure is fine, but having changes in cache that are not yet reflected on disk is, OK, I'll say it: STUPID. (Note that I'm _only_ talking about the disk structure, not file data.) With a stale cache, if power is cut or any other malfeasance occurrs, the entire volume is at risk. FSCK, even if it works flawlessly every time, is another silly thing: The file system _should_ provide for knowledge of what files were open at the time, and a check be made only of them, where that may make sense. (Some file systems of the past have not needed _any_ fsck type of checking following system failure, and they have been quite reliable.) I remain in dumbfounded awe that Unix/Linux has come all this way with such a fundamentally flawed file system paradigm. That said, I've heard that there are some new file system types available that address my concerns. I sure wish I had time to hunt them down! It's my guess that some of you are familliar with these file systems. ...I am, (we are) running Red Hat 7.2 on most of my/our systems and would love to find a file system that makes me feel more at ease. It's rare, but does happen that a system will need a kick in the pants and has to be brought down abruptly. If the file system weren't so flawed, I'd be much less anxious at these moments! Yes, I have even turned on sync, but from a performance standpoint, it's a dumb way to go since _all_ data is written synchronously - it's only a move of desperation. Still, for some of our more critical systems, sync is left ON! That's how important this issue is. Please note that in my experience, ext3 doesn't work. When I upgraded my systems from RH 6.2 to 7.2, I tried to tell it to use ext3, the default, on several systems and each time was rewarded with a system that wouldn't boot. (I don't recall just what the errors were, unfortunately.) If you're going to propose an alternative file system, please give me some practical advice about how I'm going to apply it. Thank you for your thoughts, Richard -- Richard Troy, Chief Scientist Science Tools Corporation rtroy@ScienceTools.com, 510-567-9957, http://ScienceTools.com/ On Tue, 16 Apr 2002, Bill Crawford wrote: > Date: Tue, 16 Apr 2002 22:01:18 +0100 (BST) > From: Bill Crawford <redhat-devel@billemon.org.uk> > Reply-To: redhat-devel-list@redhat.com > To: redhat-devel-list@redhat.com > Subject: Re: XFS - here's the solution > > On 16 Apr 2002, Florin Andrei wrote: > > > On Thu, 2002-03-07 at 07:37, Knut J Bjuland wrote: > > > Are Redhat going to include XFS in a rawhide kernel or in Redhat 8.X when it is ready? > > > > I'm waiting for that since a loooong time. > > > > In my tests, XFS performed better (in terms of stability AND performance > > under heavy load) than ReiserFS and Ext3 for workloads like: databases > > (MySQL), Samba and NFS fileservers (and also including here webservers, > > 'cause they are fileservers after all, aren't they?) and access to large > > files. > > Have you tried jfs? I found it's fairly good for most things, and > the quickest of all for coming back from a crash (you *must* run fsck > on it or it won't mount, but I've yet to notice any significant delay > from the journal replay). > > > ReiserFS outperformed it (and everyone else) for proxy servers (Squid). > > Reiser was optimised for Squid a while back (someone paid for it :o) > > > Ext3 outperformed it for mail relays (strange, isn't?). > > I think the way ext3 does its journalling and writing it back improves > the performance for typical mail handling (i.e. it makes fsync fast). > > > For general usage (regular workstations), there are no notable > > differences between filesystems. > > In general use, the effect of caching in memory mitigates well in > your favour; you're rarely exposed to raw filesystem performance as > such. That's a good thing as all filesystems have some weaknesses > (simply because some things are tradeoffs; sometimes the tradeoff is > selectable by the user e.g. space efficiency -vs- speed on ReiserFS > using the "notail" option). > > I'd suggest time trialling each of your major uses and choose file > system types for each system partition based on the results. You'll > probably end up with different choices to mine ;o) > > > As a matter of fact, i'm using the XFS version of Red Hat 7.2 > > everywhere, except laptops and the workloads mentioned above (proxies, > > mail relays). > > I've seen a case when, by simply changing the kernel from the regular > > one to the XFS-enabled one, the system load decreased noticeably, and > > the I/O activity became more relaxed (a very busy webserver). > > Databases too, perform usually better on XFS than on other filesystems > > (only tried MySQL). > > I also found the "system" load (measured as CPU percentage) very low > with jfs, which is currently available as part of the Raw Hide kernel, > I believe the kernel support is in the beta kernel as well but they > didn't include the jfsutils package which you will need for real life > use ... you *need* fsck.jfs in case of an unclean shutdown, but as I > noted it's not a big timewaster). > > The kernel changelog notes that jfs was included "for evaluation" so > you would need to check with RH's kernel team if they are confident of > including it for the foreseeable [sic] future. > > > > > _______________________________________________ > Redhat-devel-list mailing list > Redhat-devel-list@redhat.com > https://listman.redhat.com/mailman/listinfo/redhat-devel-list >
