I create 2-4 RBD images sized 10GB or more with --thick-provision, then
run
fio -ioengine=rbd -direct=1 -invalidate=1 -name=test -bs=4k -iodepth=128
-rw=randwrite -pool=rpool -runtime=60 -rbdname=testimg
For each of them at the same time.
How do you test what total 4Kb random write iops (RBD) you have?
-----Original Message-----
From: Vitaliy Filippov [mailto:vitalif@xxxxxxxxxx]
Sent: 24 February 2019 17:39
To: David Turner
Cc: ceph-users; 韦皓诚
Subject: *****SPAM***** Re: Configuration about using nvme
SSD
I've tried 4x OSD on fast SAS SSDs in a test setup with only 2 such
drives in cluster - it increased CPU consumption a lot, but total 4Kb
random write iops (RBD) only went from ~11000 to ~22000. So it was 2x
increase, but at a huge cost.
One thing that's worked for me to get more out of nvmes with Ceph is
to create multiple partitions on the nvme with an osd on each
partition.
That
way you get more osd processes and CPU per nvme device. I've heard of
people using up to 4 partitions like this.
--
With best regards,
Vitaliy Filippov
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
