On Fri, Mar 18, 2011 at 11:13:00AM +0000, Mel Gorman wrote: > To confirm if this is the case, I'd be very interested in hearing if this > problem persists in the following cases > > 1. 2.6.38-rc8 with defrag disabled by > echo never >/sys/kernel/mm/transparent_hugepage/defrag > (this will stop THP allocations calling into compaction) > 2. 2.6.38-rc8 with THP disabled by > echo never > /sys/kernel/mm/transparent_hugepage/enabled > (if the problem still persists, then page reclaim is still a problem > but we should still stop THP doing sync writes) > 3. 2.6.37 vanilla > (in case this is a new regression introduced since then) > > Migration can do sync writes on dirty pages which is why it looks so similar > to page reclaim but this can be controlled by the value of sync_migration > passed into try_to_compact_pages(). If we find that option 1 above makes > the regression go away or at least helps a lot, then a reasonable fix may > be to never set sync_migration if __GFP_NO_KSWAPD which is always set for > THP allocations. I've added Andrea to the cc to see what he thinks. I agree. Forcing sync=0 when __GFP_NO_KSWAPD is set, sounds good to me, if it is proven to resolve these I/O waits. Also note that 2.6.38 upstream still misses a couple of important compaction fixes that are in aa.git (everything relevant is already queued in -mm but it was a bit late for 2.6.38), so I'd also be interested to know if you can reproduce in current aa.git origin/master branch. If it's a __GFP_NO_KSWAPD allocation (do_huge_pmd_anonymous_page()) that is present in the hanging stack traces, I strongly doubt any of the changes in aa.git is going to help at all, but it worth a try to be sure. http://git.kernel.org/?p=linux/kernel/git/andrea/aa.git;a=shortlog http://git.kernel.org/?p=linux/kernel/git/andrea/aa.git;a=commit;h=48ad57f498835621d8bad83b972ee6e6c395523a http://git.kernel.org/?p=linux/kernel/git/andrea/aa.git;a=commit;h=8f6854f7cbf71bc61758bcd92497378e1f677552 http://git.kernel.org/?p=linux/kernel/git/andrea/aa.git;a=commit;h=8ff6d16eb15d2b328bbe715fcaf453b6fedb2cf9 http://git.kernel.org/?p=linux/kernel/git/andrea/aa.git;a=commit;h=e31adb46cd8c4f331cfb02c938e88586d5846bf8 This is the implementation of Mel's idea that you can apply to upstream or aa.git to see what happens... === Subject: compaction: use async migrate for __GFP_NO_KSWAPD From: Andrea Arcangeli <aarcange@xxxxxxxxxx> __GFP_NO_KSWAPD allocations are usually very expensive and not mandatory to succeed (they have graceful fallback). Waiting for I/O in those, tends to be overkill in terms of latencies, so we can reduce their latency by disabling sync migrate. Signed-off-by: Andrea Arcangeli <aarcange@xxxxxxxxxx> --- diff --git a/mm/page_alloc.c b/mm/page_alloc.c index bd76256..36d1c79 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2085,7 +2085,7 @@ rebalance: sync_migration); if (page) goto got_pg; - sync_migration = true; + sync_migration = !(gfp_mask & __GFP_NO_KSWAPD); /* Try direct reclaim and then allocating */ page = __alloc_pages_direct_reclaim(gfp_mask, order, -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxxx For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>