Hi Sage, Unless I misunderstood something ( which is still possible at this stage ;-) decode() is used both for recovery of missing chunks and retrieval of the original buffer. Decoding the M data chunks is a special case of decoding N <= M chunks out of the M+K chunks that were produced by encode(). It can be used to recover parity chunks as well as data chunks. https://github.com/dachary/ceph/blob/wip-4929/doc/dev/osd_internals/erasure-code.rst#erasure-code-library-abstract-api map<int, buffer> decode(const set<int> &want_to_read, const map<int, buffer> &chunks) decode chunks to read the content of the want_to_read chunks and return a map associating the chunk number with its decoded content. For instance, in the simplest case M=2,K=1 for an encoded payload of data A and B with parity Z, calling decode([1,2], { 1 => 'A', 2 => 'B', 3 => 'Z' }) => { 1 => 'A', 2 => 'B' } If however, the chunk B is to be read but is missing it will be: decode([2], { 1 => 'A', 3 => 'Z' }) => { 2 => 'B' } Cheers On 18/08/2013 19:34, Sage Weil wrote: > On Sun, 18 Aug 2013, Loic Dachary wrote: >> Hi Ceph, >> >> I've implemented a draft of the Erasure Code plugin loader in the context of http://tracker.ceph.com/issues/5878. It has a trivial unit test and an example plugin. It would be great if someone could do a quick review. The general idea is that the erasure code pool calls something like: >> >> ErasureCodePlugin::factory(&erasure_code, "example", parameters) >> >> as shown at >> >> https://github.com/ceph/ceph/blob/5a2b1d66ae17b78addc14fee68c73985412f3c8c/src/test/osd/TestErasureCode.cc#L28 >> >> to get an object implementing the interface >> >> https://github.com/ceph/ceph/blob/5a2b1d66ae17b78addc14fee68c73985412f3c8c/src/osd/ErasureCodeInterface.h >> >> which matches the proposal described at >> >> https://github.com/dachary/ceph/blob/wip-4929/doc/dev/osd_internals/erasure-code.rst#erasure-code-library-abstract-api >> >> The draft is at >> >> https://github.com/ceph/ceph/commit/5a2b1d66ae17b78addc14fee68c73985412f3c8c >> >> Thanks in advance :-) > > I haven't been following this discussion too closely, but taking a look > now, the first 3 make sense, but > > virtual map<int, bufferptr> decode(const set<int> &want_to_read, const > map<int, bufferptr> &chunks) = 0; > > it seems like this one should be more like > > virtual int decode(const map<int, bufferptr> &chunks, bufferlist *out); > > As in, you'd decode the chunks you have to get the actual data. If you > want to get (missing) chunks for recovery, you'd do > > minimum_to_decode(...); // see what we need > <fetch those chunks from other nodes> > decode(...); // reconstruct original buffer > encode(...); // encode missing chunks from original data > > sage > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Loïc Dachary, Artisan Logiciel Libre All that is necessary for the triumph of evil is that good people do nothing.
Attachment:
signature.asc
Description: OpenPGP digital signature
