Hi Andreas, I spent some time this week trying to figure out something that would be reasonably generic, readable from the sysadmin point of view and simple to implement. The input of the plugin is here: http://tracker.ceph.com/issues/7146#note-1 The json structure describes the pyramid and associates an erasure code method with each layer, including parameters. The mapping describes how chunks relate to the list of OSDs obtained from crush. For instance in |^000111^| the | are ignored ( whitespace is confusing because it's not easy to figure out visually how many of them there are ), ^ marks a coding chunk, any other character is a data chunk. The pyramid encoding function reads this and encode the first three data chunks with one coding chunk. The re-ordering of the chunks is done by the pyramid code and the underlying erasure code method does not need to know anything about it. There is no copy involved, it re-orders pointers ( bufferptr ). Here is a draft (not compiling not working but the logic looks right to me) implementation: encode : https://github.com/dachary/ceph/blob/wip-pyramid/src/osd/ErasureCodePluginPyramid/ErasureCodePyramid.cc#L250 decode : https://github.com/dachary/ceph/blob/wip-pyramid/src/osd/ErasureCodePluginPyramid/ErasureCodePyramid.cc#L367 The plugins for each layer would be loaded at init time : https://github.com/dachary/ceph/blob/wip-pyramid/src/osd/ErasureCodePluginPyramid/ErasureCodePyramid.cc#L83 with as much consistency checks as possible, for instance: https://github.com/dachary/ceph/blob/wip-pyramid/src/osd/ErasureCodePluginPyramid/ErasureCodePyramid.cc#L102 so that runtime can assume constraints are enforced. Please let me know if you see something that does not look right, this is a draft, it can be reworked 100% ;-) Cheers -- Loïc Dachary, Artisan Logiciel Libre
Attachment:
signature.asc
Description: OpenPGP digital signature
