Hi Ceph, For the record, here are a few erasure code related links (thanks Kevin :-), https://bitbucket.org/tsg-/liberasurecode/ liberasurecode is an Erasure Code API library written in C with pluggable Erasure Code backends. the first commit is dated back april 2014 but its content suggests development happened elsewhere before that. It is the equivalent of a standalone https://github.com/ceph/ceph/tree/master/src/erasure-code Only with dynamically loaded jerasure/isa instead of an embedded copy. The idea of an external general purpose erasure code library is appealing. It would lower the maintenance effort, among other things. It's nice to see that Ceph is listed as a contributor in the readme. There is no commit to date in the liberasurecode library itself but that can be fixed ;-) liberasurecode is already used as a dependency of https://bitbucket.org/kmgreen2/pyeclib , the library swift depends on for erasure coded storage policies. And some interesting papers: Simple Regenerating Codes: http://arxiv.org/pdf/1109.0264v1.pdf LT Codes: http://en.wikipedia.org/wiki/Luby_transform_code Rateless codes and downloads: http://www.iptps.org/papers-2003/rateless_codes.pdf Yesterday I attended Paul Luse presentation of the erasure code implementation in swift and it was enlightening. Very much like Samuel Just did in Ceph when he introduced the Placement Groups backends, Paul worked on Swift Storage Policies to make room for erasure coded "rings" (pools in the swift parlance, if I'm not mistaken). http://docs-draft.openstack.org/90/125190/8/gate/gate-swift-specs-docs/2d2cf67/doc/build/html/specs/swift/erasure_coding.html It looks like erasure code will be available in swift soon and the hard part of the work is complete. Cheers -- Loïc Dachary, Artisan Logiciel Libre
Attachment:
signature.asc
Description: OpenPGP digital signature
