On 4/3/2013 10:31 AM, Peter Landmann wrote: > Stan Hoeppner <stan <at> hardwarefreak.com> writes: > >> >> On 4/3/2013 6:00 AM, Peter Landmann wrote: >> >> You didn't mention your stripe_cache_size value. It'll make a lot of >> difference. Make sure it's at least 4096. The default is 256. > > You are very right. > I increased it to 4096 - 32768 and the performance increased much. Be careful here. Increasing stripe_cache_size increases memory consumption of md dramatically. Formula: stripe_cache_size * 4096 bytes * drive_count = RAM usage. For a 6 drive array that's stripe_cache_size RAM consumed 4096 96MB 8192 192MB 16384 384MB 32768 768MB Thus you want to select a value that gives you the best combination of performance and lowest memory usage, unless you're not concerned about RAM. > Also i played a bit with deadline parameters and it helped also to increase > performance. ... > With Raid 5 and 6 SSDs i got 33936 IOPS (fio settings as before) which is not > far away from theoretical 40000 (i know from former tests that the performance > could be increased for some more jobs). Always test with parallel threads. If you don't you're not getting a realistic picture of what md/RAID and the hardware are capable of. > For your info: With Raid 6 and 6 SSDs i got 32526 IOPS which is also a very good > result. > > So i conclude that there is no (big) problem with scalability at this hw level, > right? Yes. What this demonstrates is that one Thuban core at 2.8-3.3GHz can apparently execute the md/RAID5/6 write threads faster than these 6 X25-M G2 SSDs can sink the writes. If your CPU was a 1.6GHz Atom and/or these were newer SATAIII Sandforce based SSDs, you'd peak a CPU core long before the SSDs run out of headroom. > FYI: The scheduler makes the difference. If you alternate writes and reades in > small steps (R W R R W R W W R ..) then the performce decreases heavily. If you > group read and write operations (20xW 20xR 20xW ..)then the performance will be > better. Tested it without raid and a patched fio (and noop scheduler). But > deadline scheduler can reach the same i learned. The scheduler can play a difference, but with SSDs noop usually gives the best results. With some SATA/drive controller combos deadline may be better. CFQ is rarely, if ever, good for performance. > Thx for your informations and hints You bet. -- Stan -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
