On Fri, May 24, 2019 at 6:26 AM Robert Sander <r.sander@xxxxxxxxxxxxxxxxxxx> wrote:
Am 24.05.19 um 14:43 schrieb Paul Emmerich:
> 20 MB/s at 4K blocks is ~5000 iops, that's 1250 IOPS per SSD (assuming
> replica 3).
>
> What we usually check in scenarios like these:
>
> * SSD model? Lots of cheap SSDs simply can't handle more than that
The system has been newly created and is not busy at all.
We tested a single SSD without OSD on top with fio: it can do 50K IOPS
read and 16K IOPS write.
You probably tested with async writes, try passing sync to fio, that is much closer to what Ceph will do as it syncs every write to make sure it is written to disk before acknowledging back to the client that the write is done. When I did these tests, I also filled the entire drive and ran the test for an hour. Most drives looked fine with short tests are small amounts of data, but once the drive started getting full, the performance dropped off a cliff. Considering that Ceph is really hard on drives, it's good to test the extreme.
Robert LeBlanc
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
