Yes, as Jason suggests - 27 IOPS doing 4k blocks is: 27*4/1024 MB/s = 0.1 MB/s While the RBD volume is composed of 4MB objects - many of the (presumably) random IOs of 4k blocks can reside in the same 4MB object, so it is tricky to estimate how many 4MB objects are needing to be rewritten each second (i.e your write amplification). Generally 27 IOPS is not particularly good, even with a client connecting via 1Gbit (after all *it* can send or receive the 4k blocks at a max of about 100MB/s) [1]. To be able to provide any more useful help, we'd need to know a bit more about your ceph cluster (version, how many machines, osd's, ceph.conf) and how your client is setup (vm using librbd or kernel rbd etc). Also your fio job file would be handy to see. Cheers Mark [1] I guess it might be worthwhile to check that the client *can* actually achieve 100MB/s, if not there is some networking glitch somewhere in your system that needs to be fixed before you retest. On 01/09/14 14:51, Jason King wrote: > Guess you should multiply 27 by bs=4k? > > Jason > > > 2014-08-29 15:52 GMT+08:00 lixuehui at chinacloud.com.cn > <mailto:lixuehui at chinacloud.com.cn> <lixuehui at chinacloud.com.cn > <mailto:lixuehui at chinacloud.com.cn>>: > > > guys: > There's a ceph cluster working and nodes were connected with > 10Gb cable. We defined fio's bs=4k and the object size of rbd is > 4MB. > Client node was connect with the cluster via 1000Mb cable. Finally > the IOPS is 27 ,when we control the latency of fio under 20ms. > 4MB*27=108MB nearly 1000Mb,so is it reasonable?
 |