Re: performance in a small cluster

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On Tue, Jun 4, 2019 at 1:39 PM <vitalif@xxxxxxxxxx> wrote:
>> Basically they max out at around 1000 IOPS and report 100%
>> utilization and feel slow.
>>
>> Haven't seen the 5200 yet.

Micron 5100s performs wonderfully!

You have to just turn its write cache off:

hdparm -W 0 /dev/sdX

1000 IOPS means you haven't done it. Although even with write cache
enabled I observe like ~5000 iops, not 1000, but that delta is probably
just eaten by Ceph :))

can confirm that there are several disks where the write cache seems to
be broken and this helps a lot. Good to know that this is one of these
disks.

(The cluster with these disks that I've worked on didn't need more than
a few IOPS per disk and had other problems, so I didn't check it there)


Paul
 

With write cache turned off 5100 is capable of up to 40000 write iops.
5200 is slightly worse, but only slightly: it still gives ~25000 iops.

Funny thing is that the same applies to a lot of server SSDs with
supercapacitors. As I understand when their write cache is turned on
every `fsync` is translated to SATA FLUSH CACHE, and the latter is
interpreted by the drive as "please flush all caches, including
capacitor-protected write cache".

And when you turn it off the drive just writes at its full speed and
doesn't flush the cache because it has capacitors to account for a
possible power loss.

You don't need to disable cache explicitly only with some HBAs that do
it internally.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux