On 12/10/2015 04:14 PM, Dallas Clement wrote: > Exactly. I'm not expecting RMWs to be happening for large sequential > writes. But yet my RAID 5, 6 sequential write performance is still > very poor. As mentioned earlier, I'm getting around 95 MB/s on the > inner side of these disks. With 12 of them, my RAID 6 write speed > should be (12 - 2) * 95 = 950 MB/s. I'm getting about 300 MB/s less > than that for this scenario. I have the disks split up among three > different controllers. There should be plenty of bandwidth. Several > days ago I ran fio on each of the 12 disks concurrently. I was able > to see the disks at or near 100% utilization and wMB/s around 160-170 > MB/s. That's why I started focusing on RAID as being the potential > bottleneck. > >> That's why I questioned O_SYNC when you were using a filesystem: it >> prevents merging, and forces seeking to do small metadata writes. >> Basically turning a sequential workload into a random one. > > Yes, that certainly makes sense. Not using O_SYNC anymore. Just O_DIRECT. Sounds like its time to break out blktrace to see what's really happening between your array and its member devices. With diffs from old kernels to new. Phil -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
