On 12/11/2015 09:55 PM, Dallas Clement wrote: > Right. I understand the fio iodepth is different than the hardware > queue depth. But the fio man page seems to only mention limitation on > synchronous operations which mine are not. I'm using direct=1 and > sync=0. You are confusing sequential and synchronous. The man page says it is ineffective for *sequential* operations, especially when direct=1. > I guess what I would really like to know is how I can achieve at or > near 100% utilization on the raid device and its member disks with > fio. Do I need to increase /sys/block/sd*/device/queue_depth and > /sys/block/sd*/queue/nr_requests to get more utilization? I don't know specifically. It seems to me that increasing queue depth adds resiliency in the face of data transfer timing jitter, but at the cost of more CPU overhead. I'm not convinced fio is the right workload, either. It's options are much more flexible for random I/O workloads. dd isn't perfect either, especially when writing zeroes -- it actually reads zeros over and over from the special device. For sequential operations I like dc3dd with its pat= wipe= mode. That'll only generate writes. >> That's why I suggested blktrace. Collect a trace while a single dd is >> writing to your raw array device. Compare the large writes submitted to >> the md device against the broken down writes submitted to the member >> devices. > > Sounds good. Will do. What signs of trouble should I be looking for? Look for strictly increasing logical block addresses in requests to the member devices. Any disruption in that will break optimum positioning for streaming throughput. Per device. Requests to the device have to be large enough and paced quickly enough to avoid starving the write head. Of course, any reads mixed in mean RMW cycles you didn't avoid. You shouldn't have any of those for sequential writes in chunk * (n-2) multiples. I know it's a bit hand-wavy, but you have more hardware to play with than I do :-) Phil -- To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
