in most cases write back cache does help a lot for hdd write latency,
either raid-0 or some Areca cards support write back in jbod mode. Our
observation they could help by a 3-5x factor in Bluestore, whereas
db/wal on flash will be about 2x, it does depend on hardware but in
general we see benefit in combining both if hdds are used. With
Filestore the effect of journal on flash was better than 2x in terms of
latency/iops.
/Maged
On 17/07/2019 15:00, Mark Nelson wrote:
Some of the first performance studies we did back at Inktank were
looking at RAID-0 vs JBOD setups! :) You are absolutely right that
the controller cache (especially write-back with a battery or
supercap) can help with HDD-only configurations. Where we typically
saw problems was when you load up a chassis with lots of drives and
use SAS expanders with a single controller. In some cases we saw
higher tail latency and of course you can hit throughput limitations
for large IOs too with enough disks. This happens much quicker if
you've got SSDs in the mix for DB/WAL too. Back then, the question
really was whether you were better off investing money in a
controller+cache or jbod-only setup with flash journals (now DB/WAL).
I'm guessing it's still worth prioritizing flash over the controller,
but if you've already got the controller it may not be a bad idea to
use single-disk RAID0 depending on your use case. JBOD does make
system management a bit more friendly imho.
Regarding Disk Cache: We've diagnosed some very strange disk cache
behavior with customers in the past. Nothing recent though.
Mark
On 7/17/19 7:27 AM, John Petrini wrote:
Dell has a whitepaper that compares Ceph performance using JBOD and
RAID-0 per disk that recommends RAID-0 for HDD's:
en.community.dell.com/techcenter/cloud/m/dell_cloud_resources/20442913/download
<http://en.community.dell.com/techcenter/cloud/m/dell_cloud_resources/20442913/download>
After switching from JBOD to RAID-0 we saw a huge reduction in
latency, the difference was much more significant than that
whitepaper shows. RAID-0 allows us to leverage the controller cache
which has major performance improvements when used with HDD's. We
also disable the disk cache on our HDD's and SSD's as we had
inconsistent performance with disk cache enabled.
As always I'd suggest testing various configurations with your own
hardware but I wouldn't shy away from RAID-0 simply because of "best
practice".
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
