On Tue, Jun 4, 2019 at 1:39 PM <vitalif@xxxxxxxxxx> wrote:
>> Basically they max out at around 1000 IOPS and report 100%
>> utilization and feel slow.
>>
>> Haven't seen the 5200 yet.
Micron 5100s performs wonderfully!
You have to just turn its write cache off:
hdparm -W 0 /dev/sdX
1000 IOPS means you haven't done it. Although even with write cache
enabled I observe like ~5000 iops, not 1000, but that delta is probably
just eaten by Ceph :))
can confirm that there are several disks where the write cache seems to
be broken and this helps a lot. Good to know that this is one of these
disks.
(The cluster with these disks that I've worked on didn't need more than
a few IOPS per disk and had other problems, so I didn't check it there)
Paul
With write cache turned off 5100 is capable of up to 40000 write iops.
5200 is slightly worse, but only slightly: it still gives ~25000 iops.
Funny thing is that the same applies to a lot of server SSDs with
supercapacitors. As I understand when their write cache is turned on
every `fsync` is translated to SATA FLUSH CACHE, and the latter is
interpreted by the drive as "please flush all caches, including
capacitor-protected write cache".
And when you turn it off the drive just writes at its full speed and
doesn't flush the cache because it has capacitors to account for a
possible power loss.
You don't need to disable cache explicitly only with some HBAs that do
it internally.
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
