On Tue, Jun 22, 2010 at 04:38:56PM +0200, Jan Kara wrote: > On Tue 22-06-10 10:31:24, Christoph Hellwig wrote: > > On Tue, Jun 22, 2010 at 09:52:34PM +0800, Wu Fengguang wrote: > > > 2) most writeback will be submitted by one per-bdi-flusher, so no worry > > > of cache bouncing (this also means the per CPU counter error is > > > normally bounded by the batch size) > > > > What counter are we talking about exactly? Once balanance_dirty_pages > The new per-bdi counter I'd like to introduce. > > > stops submitting I/O the per-bdi flusher thread will in fact be > > the only thing submitting writeback, unless you count direct invocations > > of writeback_single_inode. > Yes, I agree that the per-bdi flusher thread should be the only thread > submitting lots of IO (there is direct reclaim or kswapd if we change > direct reclaim but those should be negligible). So does this mean that > also I/O completions will be local to the CPU running per-bdi flusher > thread? Because the counter is incremented from the I/O completion > callback. By default we set QUEUE_FLAG_SAME_COMP, which means we hand completions back to the submitter CPU during blk_complete_request(). Completion processing is then handled by a softirq on the CPU selected for completion processing. This was done, IIRC, because it provided some OLTP benchmark 1-2% better results. It can, however, be turned off via /sys/block/<foo>/queue/rq_affinity, and there's no guarantee that the completion processing doesn't get handled off to some other CPU (e.g. via a workqueue) so we cannot rely on this completion behaviour to avoid cacheline bouncing. Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxxx For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>