FWIW, those guidelines try to be sort of a one-size-fits-all
recommendation that may not apply to your situation. Typically RBD has
pretty low metadata overhead so you can get away with smaller DB
partitions. 4% should easily be enough. If you are running heavy RGW
write workloads with small objects, you will almost certainly use more
than 4% for metadata (I've seen worst case up to 50%, but that was
before column family sharding which should help to some extent). Having
said that, bluestore will roll the higher rocksdb levels over to the
slow device and keep the wall, L0, and other lower LSM levels on the
fast device. It's not necessarily the end of the world if you end up
with some of the more rarely used metadata on the HDD but having it on
flash certain is nice.
Mark
On 6/3/21 5:18 PM, Dave Hall wrote:
Anthony,
I had recently found a reference in the Ceph docs that indicated something
like 40GB per TB for WAL+DB space. For a 12TB HDD that comes out to
480GB. If this is no longer the guideline I'd be glad to save a couple
dollars.
-Dave
--
Dave Hall
Binghamton University
kdhall@xxxxxxxxxxxxxx
On Thu, Jun 3, 2021 at 6:10 PM Anthony D'Atri <anthony.datri@xxxxxxxxx>
wrote:
Agreed. I think oh …. maybe 15-20 years ago there was often a wider
difference between SAS and SATA drives, but with modern queuing etc. my
sense is that there is less of an advantage. Seek and rotational latency I
suspect dwarf interface differences wrt performance. The HBA may be a
bigger bottleneck (and way more trouble).
500 GB NVMe seems like a lot per HDD, are you using that as WAL+DB with
RGW, or as dmcache or something?
Depending on your constraints, QLC flash might be more competitive than
you think ;)
— aad
I suspect the behavior of the controller and the behavior of the drive
firmware will end up mattering more than SAS vs SATA. As always it's best
if you can test it first before committing to buying a pile of them.
Historically I have seen SATA drives that have performed well as far as
HDDs go though.
Mark
On 6/3/21 4:25 PM, Dave Hall wrote:
Hello,
We're planning another batch of OSD nodes for our cluster. Our prior
nodes
have been 8 x 12TB SAS drives plus 500GB NVMe per HDD. Due to market
circumstances and the shortage of drives those 12TB SAS drives are in
short
supply.
Our integrator has offered an option of 8 x 14TB SATA drives (still
Enterprise). For Ceph, will the switch to SATA carry a performance
difference that I should be concerned about?
Thanks.
-Dave
--
Dave Hall
Binghamton University
kdhall@xxxxxxxxxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
