zhong jiang wrote: > On 2016/9/14 19:29, Tetsuo Handa wrote: > > On 2016/09/14 18:25, zhong jiang wrote: > >> On 2016/9/14 16:52, Michal Hocko wrote: > >>> On Wed 14-09-16 10:42:19, Michal Hocko wrote: > >>>> [Let's CC Hugh] > >>> now for real... > >>> > >>>> On Wed 14-09-16 15:13:50, zhong jiang wrote: > >>>> [...] > >>>>> hi, Michal > >>>>> > >>>>> Recently, I hit the same issue when run a OOM case of the LTP and ksm enable. > >>>>> > >>>>> [ 601.937145] Call trace: > >>>>> [ 601.939600] [<ffffffc000086a88>] __switch_to+0x74/0x8c > >>>>> [ 601.944760] [<ffffffc000a1bae0>] __schedule+0x23c/0x7bc > >>>>> [ 601.950007] [<ffffffc000a1c09c>] schedule+0x3c/0x94 > >>>>> [ 601.954907] [<ffffffc000a1eb84>] rwsem_down_write_failed+0x214/0x350 > >>>>> [ 601.961289] [<ffffffc000a1e32c>] down_write+0x64/0x80 > >>>>> [ 601.966363] [<ffffffc00021f794>] __ksm_exit+0x90/0x19c > >>>>> [ 601.971523] [<ffffffc0000be650>] mmput+0x118/0x11c > >>>>> [ 601.976335] [<ffffffc0000c3ec4>] do_exit+0x2dc/0xa74 > >>>>> [ 601.981321] [<ffffffc0000c46f8>] do_group_exit+0x4c/0xe4 > >>>>> [ 601.986656] [<ffffffc0000d0f34>] get_signal+0x444/0x5e0 > >>>>> [ 601.991904] [<ffffffc000089fcc>] do_signal+0x1d8/0x450 > >>>>> [ 601.997065] [<ffffffc00008a35c>] do_notify_resume+0x70/0x78 > > Please be sure to include exact kernel version (e.g. "uname -r", > > "cat /proc/version") when reporting. > > > > You are reporting a bug in 4.1-stable kernel, which was prone to > > OOM livelock because the OOM reaper is not available. > > ( http://lkml.kernel.org/r/57D8012F.7080508@xxxxxxxxxx ) > > > > I think we no longer can reproduce this bug using 4.8-rc6 (or linux-next), > > but it will be a nice thing to backport __GFP_NORETRY patch to stable > > kernels which do not have the OOM reaper. > No, OOM reaper can not solve the issue completely , As had disscussed with Michal. > The conclusion is that we need come up with a better method to fix it. > > Thanks > zhongjiang I still think we no longer can reproduce this bug using 4.8-rc6 (or linux-next). As of 4.1-stable, this bug caused OOM livelock situation because TIF_MEMDIE was cleared only after returning from mmput() from exit_mm() from do_exit(). Since there is a TIF_MEMDIE thread waiting at mmput() exists, the OOM killer does not select next OOM victim because oom_scan_process_thread() returns OOM_SCAN_ABORT, although ksmd is waking up the OOM killer via a __GFP_FS allocation request. As of 4.8-rc6, the OOM reaper cannot take mmap_sem for read at __oom_reap_task() because of TIF_MEMDIE thread waiting at ksm_exit() from __mmput() from mmput() from exit_mm() from do_exit(). Thus, __oom_reap_task() returns false and oom_reap_task() will emit "oom_reaper: unable to reap pid:%d (%s)\n" message. Then, oom_reap_task() clears TIF_MEMDIE from that thread, which in turn makes oom_scan_process_thread() not to return OOM_SCAN_ABORT because atomic_read(&task->signal->oom_victims) becomes 0 due to exit_oom_victim() by the OOM reaper. Then, the OOM killer selects next OOM victim because ksmd is waking up the OOM killer via a __GFP_FS allocation request. Thus, this bug will be completely solved (at the cost of selecting next OOM victim) as of 4.8-rc6. > >> Adding the __GFP_NORETRY, the issue also can fixed. > >> Therefore, we can assure that the case of LTP will leads to the endless looping. > >> > >> index d45a0a1..03fb67b 100644 > >> --- a/mm/ksm.c > >> +++ b/mm/ksm.c > >> @@ -283,7 +283,7 @@ static inline struct rmap_item *alloc_rmap_item(void) > >> { > >> struct rmap_item *rmap_item; > >> > >> - rmap_item = kmem_cache_zalloc(rmap_item_cache, GFP_KERNEL); > >> + rmap_item = kmem_cache_zalloc(rmap_item_cache, GFP_KERNEL | __GFP_NORETRY); > >> if (rmap_item) > >> ksm_rmap_items++; > >> return rmap_item; > > Your patch to add __GFP_NORETRY and __GFP_NOWARN is OK. But please explicitly state that you hit this bug in 4.1-stable. Also, your trace is not only a hung task but also an OOM livelock; the kernel as of 4.1-stable is silent when OOM livelock situation occurred. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>