On Wed, 2019-02-13 at 15:40 +0000, Jean De Gyns wrote: > Greetings ! > > I created a MD RAID6 with a 512KiB chunk size out of 12 8TB drives, no internal bitmap and no journal on quad xeon gold 6154 running kernel 4.18 (Ubuntu 18.04.1) and set FIO to do a 1TiB sequential write to the device with a block size of 5M, 3 processes and a QD of 64. > > Each drive being able to achieve 215MiB/s at the beginning of the drive, I expected the output to be somewhere around the 2GiB/s mark at the beginning of the raid array. > After setting stripe_cache_size to 32768 and group_thread_cnt to 2, I only got an average 1.4GiB/s out of my array and the throughput wasn't very stable. > > I did the same test against a hardware raid controller, the Broadcom MegaRAID 9480-8i8e, and it managed a nice flat 1.9 GiB/s. > > I expected a modern cpu to easily win over a hardware controller but that wasn't the case. > Am I missing something ? At a wag... the 4GB ram cache on the raid card causing it to appear as if the disk access is faster? I have to be honest, I've long since given up trying to test the performance of raid formats/layouts/chunks/etc... due to the multiple ways the system can "do stuff" that changes the results with even the exact same manual style tests. Then again, my workloads tend to be "good enough, is good enough". I guess, however, someone needing a high speed file server bonded 10Gb links to multiple workstations running video file editing software would be a whole different ballgame. > > Many Thanks ! > > JDG
