Den ons 5 feb. 2020 kl 11:14 skrev quexian da <daquexian566@xxxxxxxxx>: > Hello, > > I'm a beginner on ceph. I set up three ceph clusters on google cloud. > Cluster1 has three nodes and each node has three disks. Cluster2 has three > nodes and each node has two disks. Cluster3 has five nodes and each node > has five disks. > All disks are HDD. Disk speed shown by `dd if=/dev/zero of=here bs=1G > count=1 oflag=direct` is 117MB/s. > > The write performance (shown by rados bench -p scbench 1000 write) > before and after the drop are: > > cluster1: 297MB/s 94.5MB/s > cluster2: 304MB/s 67.4MB/s > cluster3: 494MB/s 267.6MB/s > > It looks like the performance before the drop is nodes_num * 100MB/s, and > the performance after the drop is about osds_num * 10MB/s. I have no idea > why there is such a drop and why the performances before the drop are > linear with nodes_num. > You are probably seeing write caching up to some point when buffer ram is expired, then get down to more "real" disk speeds, minus overhead from ceph. This is why more hosts seem to give more performance in the warmup. (and of course, this applies to physical boxes too, whatever write caches are in use will be multiplied by the amount of hosts you have which uses such caches) Do mind that ceph is not a cluster aimed at winning single-threaded write competitions, but rather having many hosts serve many clients and being able to scale that to large numbers. -- May the most significant bit of your life be positive. _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |