On Thu, Mar 08, 2012 at 03:42:52PM -0500, Jeff Moyer wrote: > > So now we're back to figuring out how to tell how long I/O will take? > If writeback is issuing random access I/Os to spinning media, you can > bet it might be a while. Today, you could lower nr_requests to some > obscenely small number to improve worst-case latency. I thought there > was some talk about improving the intelligence of writeback in this > regard, but it's a tough problem, especially given that writeback isn't > the only cook in the kitchen. ... and it gets worse if there is any kind of I/O prioritization going on via ionice(), or (as was the case in our example) I/O cgroups were being used to provide proportional I/O rate controls. I don't think it's realistic to assume the writeback code can predict how long I/O will take when it does a submission. BTW, I'd have to check (having not looked at the application code in depth; the bug was primarily solved by bisection and reverting the problem commit) but I'm not entirely sure the thread doing the write was calling fsync(); the main issue as I understand things was that the application wasn't expecting the write(2) system call would block unexpectedly for long periods of time while doing small buffered, appending I/O's. (Again, for the kind of work that distributed systems do, 99th percentile latency is important!) - Ted -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
