Hi. We have a need for "bulk" storage - but with decent write latencies. Normally we would do this with a DAS with a Raid5 with 2GB Battery backed write cache in front - As cheap as possible but still getting the features of scalability of ceph. In our "first" ceph cluster we did the same - just stuffed in BBWC in the OSD nodes and we're fine - but now we're onto the next one and systems like: https://www.supermicro.com/en/products/system/1U/6119/SSG-6119P-ACR12N4L.cfm Does not support a Raid controller like that - but is branded as for "Ceph Storage Solutions". It do however support 4 NVMe slots in the front - So - some level of "tiering" using the NVMe drives should be what is "suggested" - but what do people do? What is recommeneded. I see multiple options: Ceph tiering at the "pool - layer": https://docs.ceph.com/docs/master/rados/operations/cache-tiering/ And rumors that it is "deprectated: https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/2.0/html/release_notes/deprecated_functionality Pro: Abstract layer Con: Deprecated? - Lots of warnings? Offloading the block.db on NVMe / SSD: https://docs.ceph.com/docs/mimic/rados/configuration/bluestore-config-ref/ Pro: Easy to deal with - seem heavily supported. Con: As far as I can tell - this will only benefit the metadata of the osd- not actual data. Thus a data-commit to the osd til still be dominated by the writelatency of the underlying - very slow HDD. Bcache: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-June/027713.html Pro: Closest to the BBWC mentioned above - but with way-way larger cache sizes. Con: It is hard to see if I end up being the only one on the planet using this solution. Eat it - Writes will be as slow as hitting dead-rust - anything that cannot live with that need to be entirely on SSD/NVMe. Other? Thanks for your input. Jesper _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |