On Sat, Sep 6, 2014 at 9:27 AM, Christian Balzer <chibi at gol.com> wrote: > On Sat, 06 Sep 2014 16:06:56 +0000 Scott Laird wrote: > > > Backing up slightly, have you considered RAID 5 over your SSDs? > > Practically speaking, there's no performance downside to RAID 5 when > > your devices aren't IOPS-bound. > > > > Well... > For starters with RAID5 you would loose 25% throughput in both Dan's and > my case (4 SSDs) compared to JBOD SSD journals. > In Dan's case that might not matter due to other bottlenecks, in my case > it certainly would. > It's a trade off between lower performance all the time, or much lower performance while you're backfilling those OSDs. To me, this seems like a somewhat reasonable idea for a small cluster, where losing one SSD could lose >5% of the OSDs. It doesn't seem worth the effort for a large cluster, where losing one SSD would lose < 1% of the OSDs. > > And while you're quite correct when it comes to IOPS, doing RAID5 will > either consume significant CPU resource in a software RAID case or require > a decent HW RAID controller. > > Christian I haven't worried about CPU with software RAID5 in a very long time... maybe Pentium 4 days? It's so rare to actually have 0% Idle CPU, even under high loads. Most of my RAID5 is ZFS, but the CPU hasn't been the limiting factor on my database or NFS servers. I'm even doing software crypto, without CPU support, with only a 10% performance penalty. If the CPU has AES support, crypto is free. Obviously, RAID0 (or fully parallel JBOD) will be faster than RAID5, but RAID5 is faster than RAID10 for all but the most heavily read biased workloads. Surprised the hell out of me. I'll be converting all of my database servers from RAID10 to RAIDZ. Of course, benchmarks that match your workload trump some random yahoo on the internet. :-) Ceph OSD nodes are a bit different though. They're one of the few beasts I've dealt with that are CPU, Disk, and network bound all at the same time. If you have some idle CPU during a big backfill, then I'd consider Software RAID5 a possibility. If you ever sustain 0% idle, then I wouldn't try it. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140909/85647bd2/attachment-0001.htm>
 |