Yeah, that’s your problem, doing a single thread rsync when you have quite poor write latency will not be quick. SSD journals should give you a fair performance boost, otherwise you need to coalesce the writes at the client so that Ceph is given bigger IOs at higher queue depths. RBD Cache can help here as well as potentially FS tuning to buffer more aggressively. If writeback RBD cache is enabled, data will be buffered by RBD until a sync is called by the client, so data loss can occur during this period if the app is not issuing fsyncs properly. Once a sync is called data is flushed to the journals and then later to the actual OSD store. From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Piotr Wachowicz Thanks for your answer, Nick. Typically it's a single rsync session at a time (sometimes two, but rarely more concurrently). So it's a single ~5GB typical linux filesystem from one random VM to another random VM. Apart from using RBD Cache, is there any other way to improve the overall performance of such a use case in a Ceph cluster? In theory I guess we could always tarball it, and rsync the tarball, thus effectively using sequential IO rather than random. But that's simply not feasible for us at the moment. Any other ways? Sidequestion: does using RBDCache impact the way data is stored on the client? (e.g. a write call returning after data has been written to Journal (fast) vs written all the way to the OSD data store(slow)). I'm guessing it's always the first one, regardless of whether client uses RBDCache or not, right? My logic here is that otherwise that would imply that clients can impact the way OSDs behave, which could be dangerous in some situations. Kind Regards, Piotr On Fri, May 1, 2015 at 10:59 AM, Nick Fisk <nick@xxxxxxxxxx> wrote:
|
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
