On 5/21/2012 10:20 AM, CoolCold wrote: > On Sat, May 12, 2012 at 2:28 AM, Stan Hoeppner <stan@xxxxxxxxxxxxxxxxx> wrote: >> On 5/11/2012 3:16 AM, Daniel Pocock wrote: >> > [snip] >> That's the one scenario where I abhor using md raid, as I mentioned. At >> least, a boot raid 1 pair. Using layered md raid 1 + 0, or 1 + linear >> is a great solution for many workloads. Ask me why I say raid 1 + 0 >> instead of raid 10. > So, I'm asking - why? Neil pointed out quite some time ago that the md RAID 1/5/6/10 code runs as a single kernel thread. Thus when running heavy IO workloads across many rust disks or a few SSDs, the md thread becomes CPU bound, as it can only execute on a single core, just as with any other single thread. This issue is becoming more relevant as folks move to the latest generation of server CPUs that trade clock speed for higher core count. Imagine the surprise of the op who buys a dual socket box with 2x 16 core AMD Interlagos 2.0GHz CPUs, 256GB RAM, and 32 SSDs in md RAID 10, only to find he can only get a tiny fraction of the SSD throughput. Upon investigation he finds a single md thread peaking one core while the rest are relatively idle but for the application itself. As I understand Neil's explanation, the md RAID 0 and linear code don't run as separate kernel threads, but merely pass offsets to the block layer, which is fully threaded. Thus, by layering md RAID 0 over md RAID 1 pairs, the striping load is spread over all cores. Same with linear, avoiding the single thread bottleneck. This layering can be done with any md RAID level, creating RAID50s and RAID60s, or concatenations of RAID5/6, as well as of RAID 10. And it shouldn't take anywhere near 32 modern SSDs to saturate a single 2GHz core with md RAID 10. It's likely less than 8 SSDs, which yield ~400K IOPS, but I haven't done verufication testing myself at this point. -- Stan -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
