Mark Lehrer <lehrer@xxxxxxxxx> writes: >> One server has LSI SAS3008 [0] instead of the Perc H800, >> which comes with 512MB RAM + BBU. On most servers latencies are around >> 4-12ms (average 6ms), on the system with the LSI controller we see >> 20-60ms (average 30ms) latency. > > Are these reads, writes, or a mixed workload? I would expect an > improvement in writes, but 512MB of cache isn't likely to help much on > reads with such a large data set. It's mostly write (~20MB/s), little read (1-5 MB/s) work load. This is probably due to many people using this storage for backup. > Just as a test, you could removing the battery on one of the H800s to > disable the write cache -- or else disable write caching with megaraid > or equivalent. That is certainly an interesting idea - and rereading your message and my statement above might actually explain the behaviour: - The pattern is mainly write centric, so write latency is probably the real factor - The HDD OSDs behind the raid controllers can cache / reorder writes and reduce seeks potentially So while "a raid controller" per se does probably not improve or reduce speed for ceph, "a (disk/raid) controller with a battery backed cache", might actually. In this context: is anyone here using HBAs with battery backed cache, and if yes, which controllers do you tend to use? Nico -- Sustainable and modern Infrastructures by ungleich.ch _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |