Some of the first performance studies we did back at Inktank were
looking at RAID-0 vs JBOD setups! :) You are absolutely right that the
controller cache (especially write-back with a battery or supercap) can
help with HDD-only configurations. Where we typically saw problems was
when you load up a chassis with lots of drives and use SAS expanders
with a single controller. In some cases we saw higher tail latency and
of course you can hit throughput limitations for large IOs too with
enough disks. This happens much quicker if you've got SSDs in the mix
for DB/WAL too. Back then, the question really was whether you were
better off investing money in a controller+cache or jbod-only setup with
flash journals (now DB/WAL). I'm guessing it's still worth prioritizing
flash over the controller, but if you've already got the controller it
may not be a bad idea to use single-disk RAID0 depending on your use
case. JBOD does make system management a bit more friendly imho.
Regarding Disk Cache: We've diagnosed some very strange disk cache
behavior with customers in the past. Nothing recent though.
Mark
On 7/17/19 7:27 AM, John Petrini wrote:
Dell has a whitepaper that compares Ceph performance using JBOD and
RAID-0 per disk that recommends RAID-0 for HDD's:
en.community.dell.com/techcenter/cloud/m/dell_cloud_resources/20442913/download
<http://en.community.dell.com/techcenter/cloud/m/dell_cloud_resources/20442913/download>
After switching from JBOD to RAID-0 we saw a huge reduction in
latency, the difference was much more significant than that whitepaper
shows. RAID-0 allows us to leverage the controller cache which has
major performance improvements when used with HDD's. We also disable
the disk cache on our HDD's and SSD's as we had inconsistent
performance with disk cache enabled.
As always I'd suggest testing various configurations with your own
hardware but I wouldn't shy away from RAID-0 simply because of "best
practice".
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
