Appending to an erasure coded pool

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi there,

I am developing a web application that supports browsing, uploading, downloading, moving files in Ceph Rados pool. Internally to write objects we use rados_append, as it's often too memory intensive for us to have the full file in memory to do a rados_write_full.  

We do not control our customer's Ceph installations, such as whether they use replicated pools, EC pools etc. We've found that when dealing with a EC pool, our rados_append calls return error code 95 and message "Operation not supported". 

I've had several discussions with members in the IRC chatroom regarding this, and the general consensus I've got is:
1) Use write alignment.
2) Put a replicated pool in front of the EC pool
3) EC pools have a limited feature set 

Regarding point 1), are there any actual code example for how you would handle this in the context of rados_append? I have struggled to find even one. This seems to me something that should be handled by either the API libraries, or Ceph itself, not the client trying to write some data.

Regarding point 2) This seems to be a workaround, and generally not something we want to recommend to our customers. Is it detrimental to us an EC pool without a replicated pool? What are the performance costs of doing so?

Regarding point 3) Can you point me towards resources that describe what features / abilities you lose by adopting an EC pool? 

Many thanks in advance,

James Norman




Attachment: signature.asc
Description: Message signed with OpenPGP using GPGMail

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux