Not to be too cute, but the man page for mdadm says that --write-behind is only attempted on drives marked --write-mostly. I did not see a --write-mostly in your array create statement. Also, are you trying to create a three-way-mirror or mirror the one ssd to two HDDs as stripes. If you want the latter, you need to create a raid0 array and then the raid1 array. For testing, two drives might produce fewer anomolies. Doug Dumitru EasyCo LLC On Mon, Feb 14, 2011 at 1:38 PM, Andras Korn <korn@xxxxxxxxxxxxxxxxxxxxxxx> wrote: > > Hi, > > I experimented a bit with write-mostly and write-behind and found that > write-mostly provides a very significant benefit (see below) but > write-behind seems to have no effect whatsoever. > > This is not what I expected and I wonder if I missed something. > > I built a RAID1 array from a 64GB Corsair SSD and two 7200rpm SATA hard > disks. I created xfs on the array, then benchmarked it using bonnie++, > iozone and by compiling linux 2.6.37 (with allyesconfig). > > Some interesting benchmark results follow. I used a 2.6.38-rc2 kernel for > these measurements. > > First, the stats that were identical (within a reasonable margin of error) > across all measurements: > > bonnie++ blockwise sequential write: ~110MB/s > bonnie++ blockwise sequential rewrite: ~60MB/s > bonnie++ blockwise sequential read: ~160-175MB/s > iozone read, 16k block size: ~135MB/s > kernel compilation time, user: ~5450s (*) > kernel compilation time, system: 570s (*) > > (*) I didn't measure kernel compilation times without write-mostly; I expect > they would've been worse. > > Now for some of the measurements that resulted in (to me) surprising > differences: > > Using just the SSD (so no RAID), xfs mounted with > "noatime,noikeep,attr2,logbufs=8,logbsize=256k": > > bonnie++ seeks/s: 7791 > iozone random read, 16k block size: ~46MB/s > iozone random write, 16k block size: ~44MB/s > iozone random read, 512k block size: ~130MB/s > iozone random write, 512k block size: ~140MB/s > wall clock kernel compile time: 887s > > RAID1 from two disks and one SSD, the disks set to write-behind: > > mdadm --create /dev/md/ssdraid --force --assume-clean --level=1 \ > --raid-devices=3 --bitmap=internal --bitmap-chunk=262144 \ > /dev/sdo2 --write-behind=16383 -W /dev/sd[nm]2 > > xfs mount options: > noatime,logbsize=256k,logbufs=8,noikeep,attr2,nodiratime,delaylog > > bonnie++ seeks/s: 2087 > iozone random read, 16k block size: ~43MB/s > iozone random write, 16k block size: ~3.7MB/s > iozone random read, 512k block size: ~126MB/s > iozone random write, 512k block size: ~69MB/s > wall clock kernel compile time: 936s > > (Note the drastically reduced random write performance.) > > Now the same setup, but with write-behind=0: > > bonnie++ seeks/s: 1843 > iozone random read, 16k block size: ~48MB/s > iozone random write, 16k block size: ~3.7MB/s > iozone random read, 512k block size: ~126MB/s > iozone random write, 512k block size: ~69MB/s > wall clock kernel compile time: 935s > > So, the difference between write-behind=0 and write-behind=16383 (which > seems to be the maximum) is negligible (if not imaginary). > > For reference, some results with even write-mostly disabled: > > bonnie++ seeks/s: 487.4 > iozone random read, 16k block size: ~3.7MB/s > iozone random write, 16k block size: ~3.7MB/s > iozone random read, 512k block size: ~58MB/s > iozone random write, 512k block size: ~69MB/s > > (The full result set is available from > <http://elan.rulez.org/~korn/tmp/iobench.ods>, 27k.) > > It's easy to see from the results that write-mostly does as advertised: > reads are mostly served by the SSD, so that random reads are approximately > as fast as when I only used the SSD. > > I'd have expected write-behind to increase the apparent random write > performance though, and this didn't happen (there was no measurable > difference). > > I thought maybe the iozone benchmark was too synthetic (too many writes in > too short a time, so that the buffer effect of write-behind is lost); that's > why I tried the kernel compilation, but I the raid array was as slow with > write-behind as without it. > > Any idea why write-behind doesn't seem to have an effect? > > Thanks > > Andras > > -- > Andras Korn <korn at elan.rulez.org> > Keep your ears open - but your legs crossed. > -- > To unsubscribe from this list: send the line "unsubscribe linux-raid" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Doug Dumitru EasyCo LLC -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
