The patch titled Subject: oom: don't assume that a coredumping thread will exit soon has been removed from the -mm tree. Its filename was oom-dont-assume-that-a-coredumping-thread-will-exit-soon.patch This patch was dropped because it was merged into mainline or a subsystem tree ------------------------------------------------------ From: Oleg Nesterov <oleg@xxxxxxxxxx> Subject: oom: don't assume that a coredumping thread will exit soon oom_kill.c assumes that PF_EXITING task should exit and free the memory soon. This is wrong in many ways and one important case is the coredump. A task can sleep in exit_mm() "forever" while the coredumping sub-thread can need more memory. Change the PF_EXITING checks to take SIGNAL_GROUP_COREDUMP into account, we add the new trivial helper for that. Note: this is only the first step, this patch doesn't try to solve other problems. The SIGNAL_GROUP_COREDUMP check is obviously racy, a task can participate in coredump after it was already observed in PF_EXITING state, so TIF_MEMDIE (which also blocks oom-killer) still can be wrongly set. fatal_signal_pending() can be true because of SIGNAL_GROUP_COREDUMP so out_of_memory() and mem_cgroup_out_of_memory() shouldn't blindly trust it. And even the name/usage of the new helper is confusing, an exiting thread can only free its ->mm if it is the only/last task in thread group. [akpm@xxxxxxxxxxxxxxxxxxxx: add comment] Signed-off-by: Oleg Nesterov <oleg@xxxxxxxxxx> Cc: Cong Wang <xiyou.wangcong@xxxxxxxxx> Acked-by: David Rientjes <rientjes@xxxxxxxxxx> Acked-by: Michal Hocko <mhocko@xxxxxxx> Cc: "Rafael J. Wysocki" <rjw@xxxxxxxxxxxxx> Cc: Tejun Heo <tj@xxxxxxxxxx> Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx> --- include/linux/oom.h | 11 +++++++++++ mm/memcontrol.c | 2 +- mm/oom_kill.c | 6 +++--- 3 files changed, 15 insertions(+), 4 deletions(-) diff -puN include/linux/oom.h~oom-dont-assume-that-a-coredumping-thread-will-exit-soon include/linux/oom.h --- a/include/linux/oom.h~oom-dont-assume-that-a-coredumping-thread-will-exit-soon +++ a/include/linux/oom.h @@ -92,6 +92,17 @@ static inline bool oom_gfp_allowed(gfp_t extern struct task_struct *find_lock_task_mm(struct task_struct *p); +static inline bool task_will_free_mem(struct task_struct *task) +{ + /* + * A coredumping process may sleep for an extended period in exit_mm(), + * so the oom killer cannot assume that the process will promptly exit + * and release memory. + */ + return (task->flags & PF_EXITING) && + !(task->signal->flags & SIGNAL_GROUP_COREDUMP); +} + /* sysctls */ extern int sysctl_oom_dump_tasks; extern int sysctl_oom_kill_allocating_task; diff -puN mm/memcontrol.c~oom-dont-assume-that-a-coredumping-thread-will-exit-soon mm/memcontrol.c --- a/mm/memcontrol.c~oom-dont-assume-that-a-coredumping-thread-will-exit-soon +++ a/mm/memcontrol.c @@ -1559,7 +1559,7 @@ static void mem_cgroup_out_of_memory(str * select it. The goal is to allow it to allocate so that it may * quickly exit and free its memory. */ - if (fatal_signal_pending(current) || current->flags & PF_EXITING) { + if (fatal_signal_pending(current) || task_will_free_mem(current)) { set_thread_flag(TIF_MEMDIE); return; } diff -puN mm/oom_kill.c~oom-dont-assume-that-a-coredumping-thread-will-exit-soon mm/oom_kill.c --- a/mm/oom_kill.c~oom-dont-assume-that-a-coredumping-thread-will-exit-soon +++ a/mm/oom_kill.c @@ -281,7 +281,7 @@ enum oom_scan_t oom_scan_process_thread( if (oom_task_origin(task)) return OOM_SCAN_SELECT; - if (task->flags & PF_EXITING && !force_kill) { + if (task_will_free_mem(task) && !force_kill) { /* * If this task is not being ptraced on exit, then wait for it * to finish before killing some other task unnecessarily. @@ -443,7 +443,7 @@ void oom_kill_process(struct task_struct * If the task is already exiting, don't alarm the sysadmin or kill * its children or threads, just set TIF_MEMDIE so it can die quickly */ - if (p->flags & PF_EXITING) { + if (task_will_free_mem(p)) { set_tsk_thread_flag(p, TIF_MEMDIE); put_task_struct(p); return; @@ -649,7 +649,7 @@ void out_of_memory(struct zonelist *zone * select it. The goal is to allow it to allocate so that it may * quickly exit and free its memory. */ - if (fatal_signal_pending(current) || current->flags & PF_EXITING) { + if (fatal_signal_pending(current) || task_will_free_mem(current)) { set_thread_flag(TIF_MEMDIE); return; } _ Patches currently in -mm which might be from oleg@xxxxxxxxxx are origin.patch remove-unnecessary-is_valid_nodemask.patch linux-next.patch all-arches-signal-move-restart_block-to-struct-task_struct.patch -- To unsubscribe from this list: send the line "unsubscribe mm-commits" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html