On 07/02/2019 17:07, jesper@xxxxxxxx wrote:
Hi Maged
Thanks for your reply.
6k is low as a max write iops value..even for single client. for cluster
of 3 nodes, we see from 10k to 60k write iops depending on hardware.
can you increase your threads to 64 or 128 via -t parameter
I can absolutely get it higher by increasing the parallism. But I
may have missed to explain my purpuse - I'm intested in how close to
putting local SSD/NVMe in servers I can get with RDB. Thus putting
parallel scenarios that I would never see in production in the
tests does not really help my understanding. I think a concurrency level
of 16 is in the top of what I would expect our PostgreSQL databases to do
in real life.
can you run fio with sync=1 on your disks.
can you try with noop scheduler
what is the %utilization on the disks and cpu ?
can you have more than 1 disk per node
I'll have a look at that. Thanks for the suggestion.
Jesper
Thanks for your explanation. In your case, you have low concurrency
requirements, so focusing on latency rather than total iops is your
goal. Your current setup gives 1.9 ms latency for writes and 0.6 ms for
read. These are considered good, it is difficult to go below 1 ms for
writes. As Wido pointed, to get latency down you need to insure you have
C States in your cpu settings ( or just C1 state ), you have no low
frequencies in your P States and get cpu with high GHz frequency rather
than more cores (Nick Fisk has a good presentation on this), also avoid
dual socket and NUMA. Also if money is no issue, you will get a bit
better latency with 40G or 100G network.
/Maged
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
