controlling erasure code chunk size

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



[cc' ceph-devel]

Hi Sam,

Here is how chunks are expected to be aligned:

https://github.com/ceph/ceph/blob/4c4e1d0d470beba7690d1c0e39bfd1146a25f465/src/osd/ErasureCodePluginJerasure/ErasureCodeJerasure.cc#L365

 unsigned alignment = k*w*packetsize*sizeof(int);
  if ( ((w*packetsize*sizeof(int))%LARGEST_VECTOR_WORDSIZE) )
    alignment = k*w*packetsize*LARGEST_VECTOR_WORDSIZE;
  return alignment;

If you are going to encode small objects, it may very well lead to oversized chunks if packetsize is large. At the moment the default is 3072

https://github.com/ceph/ceph/blob/4c4e1d0d470beba7690d1c0e39bfd1146a25f465/src/common/config_opts.h#L406

A value I picked when experimenting with 1MB objects encoding ( http://dachary.org/?p=2594 ). 

I'm not entirely sure why the alignment is calculated the way it is. Andreas certainly has a better understanding on this topic.

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre

Attachment: signature.asc
Description: OpenPGP digital signature


[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux