On 5/30/2011 10:24 AM, fibreraid@xxxxxxxxx wrote: > Hi All, > > I appreciate the feedback but most of it seems around File System > recommendations or to change to parity-less RAID, like RAID 10. In my > tests, there is no file system; I am testing the raw block device as I > want to establish best-numbers there before layering on the file > system. You're not performing valid taste case. You will always have a filesystem in production. The performance of every md raid level plus filesystem plus hardware combination will be different, and thus they must be tuned together, not each in isolation, especially in the case of SSDs. Case in point: slap EXT2 on your current array setup and then XFS. Test each with file based IO. You'll see XFS has radically superior parallel IO performance compared to EXT2. Tweaking the array setup will not yield significant EXT2 speedup for parallel IO. Disk striping was invented 2+ decades ago to increase performance of slow spindles for large file reads and writes, but the performance is very low for small file IO due to partial stripe width operations taking many of your spindles out of play, decreasing parallelism. Adding parity to the striping exacerbates this problem. This is the classic trade off between performance and redundancy. SSDs have no moving parts, and natively have extremely high IOPS and throughput rates, each SSD having on the order of 150x the seek rate of a mech drive, and 2-3x the streaming throughput rate. Thus, striping is irrelevant to SSD performance, and, as you've seen, will degrade small file performance due to partial width writes etc. If you truly want to maximize real world performance of those 24 SSDs, take one of your striped RAID configurations and format it with XFS using the defaults. Then run FIO with highly parallel file based IO tests, i.e. two to four worker threads per CPU core. Then delete the array and create the linear setup I previously recommended and run the same tests. When comparing the results I think you'll begin to see why I recommend this setup for both highly parallel small and large file IO. Your large file IO numbers may be a little smaller with this setup, but you should be able to play with chunk size to achieve the best balance with both small and large file IO. Regardless of chunk size, you should still see better overall parallel IOPS and throughput than with striping, especially parity striping. If you need redundancy and maximum parallel performance, and can afford to 'waste' SSDs, create 12 RAID1 devices and make a linear array of the 12, giving XFS 12 allocation groups. For parallel small file workloads this will yield better performance than RAID10 for the same cost of device space. Large file parallel performance should be similar. -- Stan -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
