> - The pattern is mainly write centric, so write latency is > probably the real factor > - The HDD OSDs behind the raid controllers can cache / reorder > writes and reduce seeks potentially OK that makes sense. Unfortunately, re-ordering HDD writes without a battery backup is kind of dangerous -- writes need to happen in order or the filesystem will punish you when you least expect it. This is the whole point of the battery backup - to make sure that out-of-order writes get written to disk even if there is a power loss in the middle of writing the controller-write-cache data in an HDD-optimized order. Your use case is ideal for an SSD-based WAL -- though it may be difficult to beat the cost of H800s these days. > In this context: is anyone here using HBAs with battery > backed cache, and if yes, which controllers do you tend to use? I almost always use MegaRAID-based controllers (such as the H800). Good luck, Mark On Tue, Apr 20, 2021 at 2:28 PM Nico Schottelius <nico.schottelius@xxxxxxxxxxx> wrote: > > > Mark Lehrer <lehrer@xxxxxxxxx> writes: > > >> One server has LSI SAS3008 [0] instead of the Perc H800, > >> which comes with 512MB RAM + BBU. On most servers latencies are around > >> 4-12ms (average 6ms), on the system with the LSI controller we see > >> 20-60ms (average 30ms) latency. > > > > Are these reads, writes, or a mixed workload? I would expect an > > improvement in writes, but 512MB of cache isn't likely to help much on > > reads with such a large data set. > > It's mostly write (~20MB/s), little read (1-5 MB/s) work load. This is > probably due to many people using this storage for backup. > > > Just as a test, you could removing the battery on one of the H800s to > > disable the write cache -- or else disable write caching with megaraid > > or equivalent. > > That is certainly an interesting idea - and rereading your message and > my statement above might actually explain the behaviour: > > - The pattern is mainly write centric, so write latency is probably the > real factor > - The HDD OSDs behind the raid controllers can cache / reorder writes > and reduce seeks potentially > > So while "a raid controller" per se does probably not improve or reduce > speed for ceph, "a (disk/raid) controller with a battery backed cache", > might actually. > > In this context: is anyone here using HBAs with battery backed cache, > and if yes, which controllers do you tend to use? > > Nico > > > -- > Sustainable and modern Infrastructures by ungleich.ch _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |