With regard to assigning files to chunks, I suggest we start by using an algorithm similar to that we use in DHT.
hash=$(cat $filename | md5sum) # convert from hex to decimal?
chunk=$((hash % number_of_chunks))
if [ x"$chunk" = x"$my_chunk_id" ]; then
bash $filename # ...and so on
fi
This is completely automatic, robust as the test set or directory structure changes (or as the number of workers changes), and should give us an approximately equal distribution among chunks.
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
