Dear Bob
Really appreciate for your review and suggestions!
Yes, your suggestion can end my infinite loop in direct_reclaim. This change I think will easier than before to mark a zone as unreclaimable right? Will it have other side effect?
I reviewed the mainline's patch list, and found below patch should be a similar case as mine, it's case is kswapd is frozen, but my case kswapd go to sleep.
>From d1908362ae0b97374eb8328fbb471576332f9fb1 Mon Sep 17 00:00:00 2001
From: Minchan Kim <minchan.kim@xxxxxxxxx>
Date: Wed, 22 Sep 2010 13:05:01 -0700
Subject: [PATCH] vmscan: check all_unreclaimable in direct reclaim path
But later below patch changed the logic, and checked the flag oom_killer_disable which seems only be set when hibernate, so my issue appeared.
>From 929bea7c714220fc76ce3f75bef9056477c28e74 Mon Sep 17 00:00:00 2001
From: KOSAKI Motohiro <kosaki.motohiro@xxxxxxxxxxxxxx>
Date: Thu, 14 Apr 2011 15:22:12 -0700
Subject: [PATCH] vmscan: all_unreclaimable() use zone->all_unreclaimable as a name
@@ -2006,13 +2002,11 @@ static bool all_unreclaimable(struct zonelist *zonelist,
continue;
if (!cpuset_zone_allowed_hardwall(zone, GFP_KERNEL))
continue;
- if (zone_reclaimable(zone)) {
- all_unreclaimable = false;
- break;
- }
+ if (!zone->all_unreclaimable)
+ return false;
}
- return all_unreclaimable;
+ return true;
}
/*
@@ -2108,6 +2102,14 @@ out:
if (sc->nr_reclaimed)
return sc->nr_reclaimed;
+ /*
+ * As hibernation is going on, kswapd is freezed so that it can't mark
+ * the zone into all_unreclaimable. Thus bypassing all_unreclaimable
+ * check.
+ */
+ if (oom_killer_disabled)
+ return 0;
Thanks!
Best Regards
Lisa Du
-----Original Message-----
From: Bob Liu [mailto:lliubbo@xxxxxxxxx]
Sent: 2013年7月24日 11:39
To: Lisa Du
Cc: linux-mm@xxxxxxxxx; Christoph Lameter; Mel Gorman
Subject: Re: Possible deadloop in direct reclaim?
On Wed, Jul 24, 2013 at 10:23 AM, Lisa Du <cldu@xxxxxxxxxxx> wrote:
> Dear Bob
> Also from my check before kswapd sleep, though nr_slab = 0 but zone_reclaimable(zone) returns true, so zone->all_unreclaimable can't be changed to 1; So even when change the nr_slab to sc->nr_reclaimed, it can't help.
>
Then the other fix might be set zone->all_unreclaimable in direct
reclaim path also, like:
@@ -2278,6 +2278,8 @@ static bool shrink_zones(struct zonelist
*zonelist, struct scan_control *sc)
}
shrink_zone(zone, sc);
+ if (sc->nr_reclaimed == 0 && !zone_reclaimable(zone))
+ zone->all_unreclaimable = 1;
}
> Thanks!
>
> Best Regards
> Lisa Du
>
>
> -----Original Message-----
> From: Lisa Du
> Sent: 2013年7月24日 9:31
> To: 'Bob Liu'
> Cc: linux-mm@xxxxxxxxx; Christoph Lameter; Mel Gorman
> Subject: RE: Possible deadloop in direct reclaim?
>
> Dear Bob
> Thank you so much for the careful review, Yes, it's a typo, I mean zone->all_unreclaimable = 0.
> You mentioned add the check in kswapd_shrink_zone(), sorry that I didn't find this function in kernel3.4 or kernel3.9.
> Is this function called in direct_reclaim?
> As I mentioned this issue happened after kswapd thread sleep, if it only called in kswapd, then I think it can't help.
>
> Thanks!
>
> Best Regards
> Lisa Du
>
>
> -----Original Message-----
> From: Bob Liu [mailto:lliubbo@xxxxxxxxx]
> Sent: 2013年7月24日 9:18
> To: Lisa Du
> Cc: linux-mm@xxxxxxxxx; Christoph Lameter; Mel Gorman
> Subject: Re: Possible deadloop in direct reclaim?
>
> On Tue, Jul 23, 2013 at 12:58 PM, Lisa Du <cldu@xxxxxxxxxxx> wrote:
>> Dear Sir:
>>
>> Currently I met a possible deadloop in direct reclaim. After run plenty of
>> the application, system run into a status that system memory is very
>> fragmentized. Like only order-0 and order-1 memory left.
>>
>> Then one process required a order-2 buffer but it enter an endless direct
>> reclaim. From my trace log, I can see this loop already over 200,000 times.
>> Kswapd was first wake up and then go back to sleep as it cannot rebalance
>> this order’s memory. But zone->all_unreclaimable remains 1.
>>
>> Though direct_reclaim every time returns no pages, but as
>> zone->all_unreclaimable = 1, so it loop again and again. Even when
>> zone->pages_scanned also becomes very large. It will block the process for
>> long time, until some watchdog thread detect this and kill this process.
>> Though it’s in __alloc_pages_slowpath, but it’s too slow right? Maybe cost
>> over 50 seconds or even more.
>
> You must be mean zone->all_unreclaimable = 0?
>
>>
>> I think it’s not as expected right? Can we also add below check in the
>> function all_unreclaimable() to terminate this loop?
>>
>>
>>
>> @@ -2355,6 +2355,8 @@ static bool all_unreclaimable(struct zonelist
>> *zonelist,
>>
>> continue;
>>
>> if (!zone->all_unreclaimable)
>>
>> return false;
>>
>> + if (sc->nr_reclaimed == 0 && !zone_reclaimable(zone))
>>
>> + return true;
>>
>
> How about replace the checking in kswapd_shrink_zone()?
>
> @@ -2824,7 +2824,7 @@ static bool kswapd_shrink_zone(struct zone *zone,
> /* Account for the number of pages attempted to reclaim */
> *nr_attempted += sc->nr_to_reclaim;
>
> - if (nr_slab == 0 && !zone_reclaimable(zone))
> + if (sc->nr_reclaimed == 0 && !zone_reclaimable(zone))
> zone->all_unreclaimable = 1;
>
> zone_clear_flag(zone, ZONE_WRITEBACK);
>
>
> I think the current check is wrong, reclaimed a slab doesn't mean
> reclaimed a page.
>
> --
> Regards,
> --Bob
--
Regards,
--Bob
��.n������g��� �a��� �&ޖ)���)��h���&������梷���� �Ǟ�m������)������^�����������v���O��zf������