On Mon, Dec 26, 2016 at 7:47 PM, Edward Shishkin <edward.shishkin@xxxxxxxxx> wrote: > > > > On 12/25/2016 02:59 AM, Dušan Čolić wrote: >> >> Fibration is a great way to decrease fragmentation and increase throughput. >> Currently there are 4 fibration plugins, lex, dot.o, ext_1 and ext_3 >> and they all have their upsides and downsides. >> >> Proposed fibration plugin combines them all so that it combines files >> with same extensions for 1, 2. 3 and 4 character extension in groups >> and sorts them in same fiber group. >> >> With this fibration plugin all eg. xvid files would be in same group >> in folder on disk sorted alphabetically > > > > What application wants all xvid files to be in the same group? > Do you have any benchmark numbers which show advantages > of the new plugin? > Xvid files are just an example. ext_1234 fibration would be equal to sum of ext_1, ext_2, ext_3, ext_4 and dot_o in one. In currently default plugin (dot_o) we sort all files by name from the start except .o files which we put at the end. So if we had a source directory with .c .h and .o files in it files by extension would be sorted like: chchchchchchchchoooooooooooooo I presumed that in some use cases it is better to have files be sorted ccccccccccchhhhhhhhhhhhhhoooooooooooo Hypothesis is to use the premise that files of same extension are in same order of size to reduce fragmentation. If we group files of same extension in groups in one directory, when we write files of same extension after deletion of some files of one extension their group would be in same order as the deleted file so they would be written in similar place and occupy the 'hole' of similar size. Ofc I am not talking about files of few kB size where Reiser4 is great at packing but about files from few MB to few GB. Eg. directory with mp3 and xvid files. mp3s are on the order of MB and xvid on the order of GB. If we sort them just by name order of xvid and mp3 files in one directory would be random so when deleting the smaller ones we would make random holes (like from mxmxmxxmmmxxxxmxxmmmx to mx xmxx mx xmx mmmx). With grouping of writing where all mp3s would be written first and all xvid after them after some deletions we would have smaller holes grouped first and larger last (like from mmmmmmmmmmmmxxxxxxxxxx to mm m mmm mmxx xxx xxx) but the main thing that after writing we would write mp3s in mp3 holes and xvid in xvid holes ergo. reduce fragmentation (like from mm m mmm mmxx xxx xxx to mmMmMMMmmmXmmxxXxxx xxx) that we would create if we would try to write xvid over mp3 holes. One obvious use case where I hypothesize that this type of fibration is better long term would be directories with content similar to usual Downloads directory, a lot of different types (and siyes) of files that get written and deleted a lot. ext_1234 fibration is the same as dot_o for directories with only one or one and .o file extension. Ofc this is just a hypothesis that I would like to prove with some fragmentation benchmarks but I wanted to hear your thoughts. And while I was looking through the code I found a part that I comprehended, elegant and easy to understand so I wanted to make something so I could learn more. > Thanks, > Edward. > Thank you for your time and effort Dushan > > >> so that we will avoid putting >> small files between them and in that way reduce fragmentation. That >> group (xvid 4 character extensions) would be among last groups under >> one directory so that all small files would be written before it. >> >> Problem with the attached patch is that currently every fibre value is >> defined as u64 (eg. static __u64 fibre_ext_3) but if I understood >> correctly comments in kassign.c and fibration.c fibration part of the >> key is only 7 bits long. >> If that is true how did fibre_ext_3 worked? >> >> Thanks >> >> Dushan > > -- To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
