At Sat, 29 May 2010 09:43:35 +0100, Gordan Bobic wrote: > > Jiro SEKIBA wrote: > > >>>> 2) Mechanical disks suffer from slow random writes (or any random > >>>> operation for that matter), too. Do the benefits of nilfs show in random > >>>> write performance on mechanical disks? > >>> I think it may have benefits, for nilfs will write sequentially whatever > >>> data is located before writing it. But still some tweaks might be required > >>> to speed up compared with ordinary filsystem like ext3. > >> Can you quantify what those tweaks may be, and when they might become > >> available/implemented? > > > > I might choose the wrong word, but what I meant is more hack is required > > to improve write performance. Not just configuration matters :(. > > I understand what you meant. I just wanted to know when those hacks may > be implemented and be available for those of us interested in using > nilfs to optimize write-heavy workloads. Nhh, that's really diffcult question. There is no target date to hack. >From kernel release point of view, at least another 3 months require to drop those kind of non-bugfix code. For, merge window has just been closed. And if you need stable release, it takes about 3 months to release after next merge window opens. That means at least a half year :(. Of course, you can chase Ryusuke's tree though. > >>>> 3) How does this affect real-world read performance if nilfs is used on > >>>> a mechanical disk? How much additional file fragmentation in absolute > >>>> terms does nilfs cause? > >>> The data is scattered if you modified the file again and again, > >>> but it'll be almost sequential at the creation time. So it will > >>> affect much if files are modified frequently. > >> Right. So bad for certain tasks, such as databases. > > > > Indeed. maybe /var type of directories too. > > Interesting. So nilfs' suitability for write heavy loads is actually > quite limited on mechanical disks, as it isn't suitable for append-heavy > situations such as databases and logging, but for use-cases that are > write+delete heavy such as mail servers or other spool type loads it > should still be advantageous. > > >>>> 4) As the data gets expired, and snapshots get deleted, this will > >>>> inevitably lead to fragmentation, which will de-linearize writes as they > >>>> have to go into whatever holes are available in the data. How does this > >>>> affect nilfs write performance? > >>> For now, my understanding, nilfs garbage collector moves the live (in use) > >>> blocks to the end of logs, so holes are not created (it is correct?). > >>> However, it leads another issue that garbage collector process, which is > >>> nilfs_cleanerd, will consume the I/O. This is major I/O performance > >>> bottle neck current implementation. > >> Since this moves files, it sounds like this could be a major issue for > >> flash media since it unnecessarily creates additional writes. Can this > >> be suppressed? > > > > You can simply kill the nilfs_clearnerd after you mount the nilfs partition. > > This case, of course, any garbage is reclaimed and finally end up with > > disk full, even size of files don't occupy the storage size. > > > > I don't have data for now, but it made about twice better write performance > > compared with "with garbage collector". > > What about enabling garbage collection, but disabling degragmentation? > De-allocating space that isn't used any more is a necessary evil, but > defragmentation is rather pointless in a lot of cases (e.g. SSDs) and > counter-productive in others (mechanical disks under heavy load). Also, > what about making the garbage collector "lazy", so that it runs either > just-in time to overwrite discarded data (worst case scenario) or runs > when the disks are idle (e.g. at ionice -c3, and even that only when > there have been no disk transactions for, some selectable number of ms)? Garbage collection and defragmentation is a set of funtcions for nilfs. Nilfs manages disk in segment basis. Each segment is 8MB (except first one). Each segments include many logs, some logs are alive, some logs are garbage. To reuse those garbage in a segment, entire segment must be freed. To do that, nilfs garbage collector moves the live logs. Therefore it's difficult to separate garbage collection and defragmentation, at least on current nilfs implementation. Still there may be better garbage collect algorithm. thanks regards, > Gordan > -- > To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > -- Jiro SEKIBA <jir@xxxxxxxxx> -- To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
