On Wed, Nov 28, 2012 at 08:43:24AM -0800, Kent Overstreet wrote: > Bunch of performance improvements and cleanups Zach Brown and I have > been working on. The code should be pretty solid at this point, though > it could of course use more review and testing. Thanks for sending these out. I have some initial review comments that'll follow, but I'm running out of steam today. I'll continue tomorrow. > The results in my testing are pretty impressive, particularly when an > ioctx is being shared between multiple threads. In my crappy synthetic > benchmark, with 4 threads submitting and one thread reaping completions, > I saw overhead in the aio code go from ~50% (mostly ioctx lock > contention) to low single digits. Performance with ioctx per thread > improved too, but I'd have to rerun those benchmarks. You should probably mention that those four threads were *spinning* on io_submit() :). I'm still guessing that this unreasonably inflated the contention amongst submitters and that without this inflation we might not find the per-cpu ioctx refcounts worth the trouble. > Performance wise, the end result of this patch series is that submitting > a kiocb writes to _no_ shared cachelines - the penalty for sharing an > ioctx is gone there. There's still going to be some cacheline contention > when we deliver the completions to the aio ringbuffer (at least if you > have interrupts being delivered on multiple cores, which for high end > stuff you do) but I have a couple more patches not in this series that > implement coalescing for that (by taking advantage of interrupt > coalescing). With that, there's basically no bottlenecks or performance > issues to speak of in the aio code. Yeah, this is good stuff. Thanks for pushing it. We should mention Jens' omnibus patch that also took on these problems: http://git.kernel.dk/?p=linux-block.git;a=commit;h=6b6723fc3e4f24dbd80526df935ca115ead578c6 - z -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
