This is a ymmv thing, it depends on one's workload. > > However, we have some questions about this and are looking for some guidance and advice. > > The first one is about the expected benefits. Before we undergo the efforts involved in the transition, we are wondering if it is even worth it. My personal sense is that it often isn't. It adds a lot of complexity for OSD lifecycle. I suspect there are many sites where if an OSD drive is replaced, it gets rebuilt without the external WAL/DB. This is one of the reasons that HDDs are a false economy. > How much of a performance boost one can expect when adding NVMe SSDs for WAL-devices to an HDD cluster? Plus, how much faster than that does it get with the DB also being on SSD. Are there rule-of-thumb number of that? Or maybe someone has done benchmarks in the past? Very workload-dependent. My limited understanding is that the WAL on faster storage only gets used for small writes, ones smaller than the deferred setting, which I think defaults to min_alloc_size. So maybe for a DB workload that does very small random overwrites? > > The second question is of more practical nature. Are there any best-practices on how to implement this? I was thinking we won't do one SSD per HDD - surely an NVMe SSD is plenty fast to handle the traffic from multiple OSDs. But what is a good ratio? Do I have one NVMe SSD per 4 HDDs? Per 6 or even 8? There was once the rule of thumb of 4 SATA HDDs per SATA SSD, 10 SATA HDDs per NVMe SSD. There isn't a hard line in the sand. The ratio is enforced in part by the chassis in use. > Also, how should I chop-up the SSD, using partitions or using LVM? Last but not least, if I have one SSD handle WAL and DB for multiple OSDs, losing that SSD means losing multiple OSDs. How do people deal with this risk? Is it generally deemed acceptable or is this something people tend to mitigate and if so how? Do I run multiple SSDs in RAID? > > I do realize that for some of these, there might not be the one perfect answer that fits all use cases. I am looking for best practices and in general just trying to avoid any obvious mistakes. This is one of the reasons why I counsel people to consider TCO -- and source hardware effectively -- so they don't get stuck with HDDs. > > Any advice is much appreciated. > > Sincerely > > Niklaus Hofer > -- > stepping stone AG > Wasserwerkgasse 7 > CH-3011 Bern > > Telefon: +41 31 332 53 63 > www.stepping-stone.ch > niklaus.hofer@xxxxxxxxxxxxxxxxx > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |