Hi community,
I have a x86_64 Chromebook running 3.14 kernel with 8G of memory. Using zram with swap size set to ~12GB. When in low memory, kswapd is awaken to reclaim pages, but under some circumstances the kernel can not find pages to reclaim while I'm sure there're still plenty of memory which could be reclaimed from background processes (For example, I run some C programs which just malloc() lots of memory and get suspended in the background. There's no reason they could't be swapped). The consequence is that most of CPU time is spent on page reclamation. The system hangs or becomes very laggy for a long period. Sometimes it even triggers a kernel panic by the hung task detector like:
<0>[46246.676366] Kernel panic - not syncing: hung_task: blocked tasks
I've added kernel message to trace the problem. I found shrink_inactive_list() can barely find any page to reclaim. More precisely, when the problem happens, lots of page have _count > 2 in __remove_mapping(). So the condition at line 662 of vmscan.c holds:
Thus the kernel fails to reclaim those pages at line 1209
It's weird that the inactive anonymous list is huge (several GB), but nothing can really be freed. So I did some hack to see if moving more pages from the active list helps. I commented out the "inactive_list_is_low()" checking at line 2420
in shrink_node_memcg() so shrink_active_list() is always called.It turns out that the hack helps. If moving more pages from the active list, kswapd works smoothly. The whole 12G zram can be used up before system enters OOM condition.
Any idea why the whole inactive anonymous LRU is occupied by pages which can not be freed for la long time (several minutes before system dies) ? Are there any parameters I can tune to help the situation ? I've tried swappiness but it doesn't help.
An alternative is to patch the kernel to call shrink_active_list() more frequently when it finds there's nothing that can be reclaimed . But I am not sure if it's the right direction. Also it's not so trivial to figure out where to add the call.
Thanks,
Cheng-Yu