small files and cluster/stripe

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



cluster/stripe will split large files across multiple volumes, but it 
seems to
always put the first part of the file on the first volume; if you have a 
bunch of small files they all end up there, and one volume gets heavily 
used by small files while the others are empty.

cluster/distribute spreads files across multiple volumes, but it puts 
the whole file on a single volume.

Some marriage of the two would be helpful for workloads which contain 
both large and small files, like adding an "option block-size ..." to 
cluster/distribute or "option distribute" to cluster/stripe; it would 
use the filename hash modulo nSubvolumes to determine which volume to 
start in for the first block, then rotate around the stripe for the rest.

I suppose I can work-around by creating multiple volumes as 
sub-directories of the same partition, then striping across those in 
rotations, and distributing across the stripes.

Is there some other way?  Am I missing something?

Jeff Anderson-Lee



[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux