Use `cache=writeback` QEMU option for HDD clusters, that should solve
your issue
Hi,
I recently upgraded my 3-node cluster to proxmox 6 / debian-10 and
recreated my ceph cluster with a new release (14.2.4 bluestore) -
basically hoping to gain some I/O speed.
The installation went flawlessly, reading is faster than before (~ 80
MB/s), however, the write speed is still really slow (~ 3,5 MB/s).
I wonder if I can do anything to speed things up?
My Hardware is as the following:
3 Nodes with Supermicro X8DTT-HIBQF Mainboard each,
2 OSD per node (2TB SATA harddisks, WDC WD2000F9YZ-0),
interconnected via Infiniband 40
The network should be reasonably fast, I measure ~ 16 GBit/s with
iperf,
so this seems fine.
I use ceph for RBD only, so my measurement is simply doing a very
simple
"dd" read and write test within a virtual machine (Debian 8) like the
following:
read:
dd if=/dev/vdb | pv | dd of=/dev/null
-> 80 MB/s
write:
dd if=/dev/zero | pv | dd of=/dev/vdb
-> 3.5 MB/s
When I do the same on the virtual machine on a disk that is on a NFS
storage, I get something about 30 MB/s for reading and writing.
If I disable the write cache on all OSD disks via "hdparm -W 0
/dev/sdX", I gain a little bit of performance, write speed is then 4.3
MB/s.
Thanks to your help from the list I plan to install a second ceph
cluster which is SSD based (Samsung PM1725b) which should be much
faster, however, I still wonder if there is any way to speed up my
harddisk based cluster?
Thank you in advance for any help,
Best Regards,
Hermann
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
