On 02/11/2014 06:10 AM, Tim Zhang wrote:
Hello, I test my ceph cluster and monitoring the disk utilization while writing into the cluster. First I test my disk can write with a speed of 60MB/s using fio (bs=64k threads=128) when formating it as xfs file system. After that I integrate it to ceph cluster. During writing data into rbd image I find the disk speed is around 10M/s (I monintor the speed using command: iostat -xm 1, and most of the time the disk utils is around 100%), I don't know why the disk is far less speed than that of using it independently. Can anyone give me some hints?
Hi Tim! There are potentially a couple of reasons for this. One, is that if you have journals on disk you're write speed will typically be halved as Ceph writes an extra copy of all data to the journal. Next, how many disks, and how much replication? What kind of write IO are you doing? Are they small? Sequential/Random? Direct IO vs buffered? Especially with small sequential writes, RBD cache helps dramatically. You'll also want to consider how much concurrency you have and the number of disks you are using in the cluster.
If you can provide a bit more information about your cluster and your test, we can likely provide a bit more feedback. Good luck!
Mark
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
