On 01/16/2015 09:19 PM, Michal Hocko wrote:
On Thu 15-01-15 22:54:20, Vinayak Menon wrote:
On 01/14/2015 10:20 PM, Michal Hocko wrote:
On Wed 14-01-15 17:06:59, Vinayak Menon wrote:
[...]
In one such instance, zone_page_state(zone, NR_ISOLATED_FILE)
had returned 14, zone_page_state(zone, NR_INACTIVE_FILE)
returned 92, and GFP_IOFS was set, and this resulted
in too_many_isolated returning true. But one of the CPU's
pageset vm_stat_diff had NR_ISOLATED_FILE as "-14". So the
actual isolated count was zero. As there weren't any more
updates to NR_ISOLATED_FILE and vmstat_update deffered work
had not been scheduled yet, 7 tasks were spinning in the
congestion wait loop for around 4 seconds, in the direct
reclaim path.
Not syncing for such a long time doesn't sound right. I am not familiar
with the vmstat syncing but sysctl_stat_interval is HZ so it should
happen much more often that every 4 seconds.
Though the interval is HZ, since the vmstat_work is declared as a
deferrable work, IIUC the timer trigger can be deferred to the next
non-defferable timer expiry on the CPU which is in idle. This results
in the vmstat syncing on an idle CPU delayed by seconds. May be in
most cases this behavior is fine, except in cases like this.
I am not sure I understand the above because CPU being idle doesn't
seem important AFAICS. Anyway I have checked the current code which has
changed quite recently by 7cc36bbddde5 (vmstat: on-demand vmstat workers
V8). Let's CC Christoph (the thread starts here:
http://thread.gmane.org/gmane.linux.kernel.mm/127229).
I will try to explain the exact observations. All the cases which I had
encountered, had similar symptoms. In one of the cases, it was CPU3
alone which had not updated the vmstat_diff. This CPU was in idle for
around 30 secs. When I looked at the tvec base for this CPU, the timer
associated with vmstat_update had its expiry time less than current
jiffies. This timer had its deferrable flag set, and was tied to the
next non-deferrable timer in the list. Since deferrable timers can't
wake up the CPU, the vmstat sync for this CPU was deferred for a long
time i.e. till the expiry of next non-deferrable timer. The issue was
caught because, one of the tasks which was in reclaim path and in the
congestion_wait loop had an associated watchdog, which resulted in a
panic after 4secs. So 4 secs is actually the watchdog expiry, and the
time we can get blocked in the congestion loop can be even more.
--
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a
member of the Code Aurora Forum, hosted by The Linux Foundation
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxx. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>