IMHO, you were probably either benchmarking the wrong thing or had a really unusual use profile. RAIDZ* always does full-stripe reads so it can verify checksums, so even small reads hit all of the devices in the vdev. That means that you get 0 parallelism on small reads, unlike most other RAID5+ systems where disks can read independently. So, with an 8-disk RAIDZ (RAIDZ2, RAIDZ3...) setup, you'll only get 1 disk worth of read IOPS, instead of 6-8 disks worth. That's a pretty massive hit. The bandwidth, availability, and general hassle of RAIDZ2 is nice, though. On Tue, May 27, 2014 at 3:49 PM, Craig Lewis <clewis at centraldesktop.com>wrote: > On 5/27/14 13:40 , phowell wrote: > > Hi > > First apologies if this is the wrong place to ask this question. > > We are running a small Ceph (0.79) cluster will about 12 osd's which are > on top of a zfs raid 1+0 (for another discussion)... which were created on > this version. > > > Just a reminder to benchmark everything, especially things you have known > to be true since the dawn of time. I benchmarked RAID10 vs. RAID5 so long > ago, I had to find a 3.5" floppy to open the spreadsheet. > > > Recently, I was testing ZFS on software encrypted volumes, and wanted to > see how badly it would impact a PostgreSQL server. My test setup was using > RAIDZ2, so I just ran the benchmark on that zpool. > > Imagine my surprise when an untuned and encrypted RAIDZ2 posted better > benchmarks than a tuned ZFS RAID10. > > > I really think the "RAID5 is bad for performance" is a nasty hold-over > from when parity calculations needed dedicated hardware. I won't be > building any more ZFS RAID10 arrays. > > > -- > > *Craig Lewis* > Senior Systems Engineer > Office +1.714.602.1309 > Email clewis at centraldesktop.com > > *Central Desktop. Work together in ways you never thought possible.* > Connect with us Website <http://www.centraldesktop.com/> | Twitter<http://www.twitter.com/centraldesktop> | > Facebook <http://www.facebook.com/CentralDesktop> | LinkedIn<http://www.linkedin.com/groups?gid=147417> | > Blog <http://cdblog.centraldesktop.com/> > > _______________________________________________ > ceph-users mailing list > ceph-users at lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140528/ce4511e2/attachment.htm>
 |