On 04/18/2013 05:19 AM, Emmanuel Lacour wrote:
Dear ceph users, I just set up a small cluster with two osds and 3 mon. (0.56.4-1~bpo70+1) OSDs are xfs (defaults mkfs options, mounted defaults,noatime) over lvm over hwraid. dd if=/dev/zero of=... bs=1M count=10000 conv=fdatasync on each ceph-* osd mounted partitions show 120MB/s on one server and 50MB/s on the second one.
It makes me a bit nervous that you are seeing such a discrepancy between the drives. Were you expecting that one server would be so much faster than the other? If a drive is is starting to fail your results may be unpredictable.
iperf between servers gives 580Mb/s I created a rbd, mapped it and did the same dd on it (direct to /dev/rbd/...). I get only 15MB/s :(
Are you doing replication? If one server has a slower drive, doing 2x replication, and you are using XFS (which tends to have some performance overhead with ceph) that might get you down into this range given than 50MB/s number you posted above. You may try connecting to the OSD admin sockets during tests and poll to see if all of the outstanding operations are backing up on one OSD.
Sebastien has a nice little tutorial on how to use the admin socket here: http://www.sebastien-han.fr/blog/2012/08/14/ceph-admin-socket/
(network interfaces shows ~ 120-150Mb/s, each server show ~30% IO wait) Any hint to increase the performance so it's not so far from non-ceph one? _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
