The patch titled Subject: mm,ksm: fix endless looping in allocating memory when ksm enable has been added to the -mm tree. Its filename is mmksm-fix-endless-looping-in-allocating-memory-when-ksm-enable.patch This patch should soon appear at http://ozlabs.org/~akpm/mmots/broken-out/mmksm-fix-endless-looping-in-allocating-memory-when-ksm-enable.patch and later at http://ozlabs.org/~akpm/mmotm/broken-out/mmksm-fix-endless-looping-in-allocating-memory-when-ksm-enable.patch Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/SubmitChecklist when testing your code *** The -mm tree is included into linux-next and is updated there every 3-4 working days ------------------------------------------------------ From: zhong jiang <zhongjiang@xxxxxxxxxx> Subject: mm,ksm: fix endless looping in allocating memory when ksm enable I hit the following hung task when runing a OOM LTP test case with 4.1 kernel. Call trace: [<ffffffc000086a88>] __switch_to+0x74/0x8c [<ffffffc000a1bae0>] __schedule+0x23c/0x7bc [<ffffffc000a1c09c>] schedule+0x3c/0x94 [<ffffffc000a1eb84>] rwsem_down_write_failed+0x214/0x350 [<ffffffc000a1e32c>] down_write+0x64/0x80 [<ffffffc00021f794>] __ksm_exit+0x90/0x19c [<ffffffc0000be650>] mmput+0x118/0x11c [<ffffffc0000c3ec4>] do_exit+0x2dc/0xa74 [<ffffffc0000c46f8>] do_group_exit+0x4c/0xe4 [<ffffffc0000d0f34>] get_signal+0x444/0x5e0 [<ffffffc000089fcc>] do_signal+0x1d8/0x450 [<ffffffc00008a35c>] do_notify_resume+0x70/0x78 The oom victim cannot terminate because it needs to take mmap_sem for write while the lock is held by ksmd for read which loops in the page allocator ksm_do_scan scan_get_next_rmap_item down_read get_next_rmap_item alloc_rmap_item #ksmd will loop permanently. There is no way forward because the oom victim cannot release any memory in 4.1 based kernel. Since 4.6 we have the oom reaper which would solve this problem because it would release the memory asynchronously. Nevertheless we can relax alloc_rmap_item requirements and use __GFP_NORETRY because the allocation failure is acceptable as ksm_do_scan would just retry later after the lock got dropped. Such a patch would be also easy to backport to older stable kernels which do not have oom_reaper. While we are at it add GFP_NOWARN so the admin doesn't have to be alarmed by the allocation failure. Link: http://lkml.kernel.org/r/1474165570-44398-1-git-send-email-zhongjiang@xxxxxxxxxx Signed-off-by: zhong jiang <zhongjiang@xxxxxxxxxx> Suggested-by: Hugh Dickins <hughd@xxxxxxxxxx> Suggested-by: Michal Hocko <mhocko@xxxxxxx> Cc: <stable@xxxxxxxxxxxxxxx> Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> --- mm/ksm.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff -puN mm/ksm.c~mmksm-fix-endless-looping-in-allocating-memory-when-ksm-enable mm/ksm.c --- a/mm/ksm.c~mmksm-fix-endless-looping-in-allocating-memory-when-ksm-enable +++ a/mm/ksm.c @@ -283,7 +283,8 @@ static inline struct rmap_item *alloc_rm { struct rmap_item *rmap_item; - rmap_item = kmem_cache_zalloc(rmap_item_cache, GFP_KERNEL); + rmap_item = kmem_cache_zalloc(rmap_item_cache, GFP_KERNEL | + __GFP_NORETRY | __GFP_NOWARN); if (rmap_item) ksm_rmap_items++; return rmap_item; _ Patches currently in -mm which might be from zhongjiang@xxxxxxxxxx are mmksm-fix-endless-looping-in-allocating-memory-when-ksm-enable.patch mm-page_owner-align-with-pageblock_nr-pages.patch mm-walk-the-zone-in-pageblock_nr_pages-steps.patch -- To unsubscribe from this list: send the line "unsubscribe stable" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html