On 24 Nov 2018, at 18.09, Anton Aleksandrov <anton@xxxxxxxxxxxxxx> wrote
We plan to have data on dedicate disk in each node and my question is
about WAL/DB for Bluestore. How bad would it be to place it on
system-consumer-SSD? How big risk is it, that everything will get
"slower than using spinning HDD for the same purpose"? And how big risk
is it, that our nodes will die, because of SSD lifespan?
just try and tell us :) I can't imagine it may be slower than colocated
db+wal+data.
also it depends on exact SSD models, but a lot of SSDs (even consumer
ones) in fact survive 10-20 times more writes than claimed by the
manufacturer. only some really cheap chinese ones don't...
there's an article on 3dnews about it: https://3dnews.ru/938764/
the real risk is the lack of power loss protection. Data can be
corrupted on unflean shutdowns
it's not! lack of "advanced power loss protection" only means lower iops
with fsync, but not the possibility of data corruption
"advanced power loss protection" is basically the synonym for
"non-volatile cache"
Disabling cache may help
it won't help on consumer ssds, because (write+fsync) performance is
roughly the same as (write with cache disabled) for them
Ceph is always issuing at least as many fsync's as writes, so it's
basically always operating in "disk cache disabled" mode
at the same time, disabling disk write cache on enterprise SSDs (hdparm -W
0) often increases random write iops by an order of magnitude. not sure
why. maybe because kernel flushes disk queue on every sync if it thinks
disk cache is enabled...
--
With best regards,
Vitaliy Filippov
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
