Re: Need help to develop CEPH EC Plugin for array type of Erasure Code

[Syed Hussain <syed789@xxxxxxxxx>]

Thanks for the follow-up.

Lets start with normal erasure coding plugin. Here we have k data node and m parity nodes.

For RDP or RAID-DP or MSR/MBR kind of erasure code the codeword is two dimensional array.

FOr example, in case of RDP Erasure code the cored is kx(k+m) array. The codeword has k rows and n columns (i.e. k + m) 

The first k columns represents the systematic data and the last m columns represents the parity data. 

Here the object to be divided into kX(k+m) symbols. The RDP Erasure code has one row parity and rests are diagonal parity. 

Pl. make google image for RDP Erasure Code diagram. 

I'm I making sense...

Thanks,

Syed

On Mon, Feb 1, 2016 at 9:03 PM, Syed Hussain <syed789@xxxxxxxxx> wrote:

Hi,

I've been working to develop a CEPH EC plgin for array type erasure code, for example

RAID-DP (i.e RDP). Later I realized that I can't continue with (k+m) format in CEPH as 

like normal RS code if I want to save disk IO (or Read data).  Since I need to ready the individual symbols from each device rather that k symbols. One "chunk" from OSD will contain k-symbols 

in contiguous form. The same thing will be an issue for MSR(Minimum Storage Regeneration) Erasure code plugin.

As a solution I thought to consider k*k data chunk and k*m coding chunk and use LRC kind of 

layering to pass the info (in plugin) which is data chunk and coding chunk.

However, I've doubt if the decoding API will supply me the correct info on lost chunks to get the info on lost erasure coding nodes.

Does anybody have good solution to this issue using CRUSH map?

Could Pl. suggest any pointer or link that explain the above issue? 

Thanks,

Syed Abid Hussain

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com