The patch titled
Subject: memcg, oom: move out_of_memory back to the charge path
has been added to the -mm tree. Its filename is
memcg-oom-move-out_of_memory-back-to-the-charge-path.patch
This patch should soon appear at
http://ozlabs.org/~akpm/mmots/broken-out/memcg-oom-move-out_of_memory-back-to-the-charge-path.patch
and later at
http://ozlabs.org/~akpm/mmotm/broken-out/memcg-oom-move-out_of_memory-back-to-the-charge-path.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next and is updated
there every 3-4 working days
------------------------------------------------------
From: Michal Hocko <mhocko@xxxxxxxx>
Subject: memcg, oom: move out_of_memory back to the charge path
3812c8c8f395 ("mm: memcg: do not trap chargers with full callstack on
OOM") has changed the ENOMEM semantic of memcg charges. Rather than
invoking the oom killer from the charging context it delays the oom killer
to the page fault path (pagefault_out_of_memory). This in turn means that
many users (e.g. slab or g-u-p) will get ENOMEM when the corresponding
memcg hits the hard limit and the memcg is is OOM. This is behavior is
inconsistent with !memcg case where the oom killer is invoked from the
allocation context and the allocator keeps retrying until it succeeds.
The difference in the behavior is user visible. mmap(MAP_POPULATE) might
result in not fully populated ranges while the mmap return code doesn't
tell that to the userspace. Random syscalls might fail with ENOMEM etc.
The primary motivation of the different memcg oom semantic was the
deadlock avoidance. Things have changed since then, though. We have an
async oom teardown by the oom reaper now and so we do not have to rely on
the victim to tear down its memory anymore. Therefore we can return to
the original semantic as long as the memcg oom killer is not handed over
to the users space.
There is still one thing to be careful about here though. If the oom
killer is not able to make any forward progress - e.g. because there is
no eligible task to kill - then we have to bail out of the charge path to
prevent from same class of deadlocks. We have basically two options here.
Either we fail the charge with ENOMEM or force the charge and allow
overcharge. The first option has been considered more harmful than useful
because rare inconsistencies in the ENOMEM behavior is hard to test for
and error prone. Basically the same reason why the page allocator doesn't
fail allocations under such conditions. The later might allow runaways
but those should be really unlikely unless somebody misconfigures the
system. E.g. allowing to migrate tasks away from the memcg to a
different unlimited memcg with move_charge_at_immigrate disabled.
Link: http://lkml.kernel.org/r/20180628151101.25307-1-mhocko@xxxxxxxxxx
Signed-off-by: Michal Hocko <mhocko@xxxxxxxx>
Acked-by: Greg Thelen <gthelen@xxxxxxxxxx>
Cc: Johannes Weiner <hannes@xxxxxxxxxxx>
Cc: Shakeel Butt <shakeelb@xxxxxxxxxx>
Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
---
diff -puN include/linux/memcontrol.h~memcg-oom-move-out_of_memory-back-to-the-charge-path include/linux/memcontrol.h
--- a/include/linux/memcontrol.h~memcg-oom-move-out_of_memory-back-to-the-charge-path
+++ a/include/linux/memcontrol.h
@@ -504,16 +504,16 @@ unsigned long mem_cgroup_get_max(struct
void mem_cgroup_print_oom_info(struct mem_cgroup *memcg,
struct task_struct *p);
-static inline void mem_cgroup_oom_enable(void)
+static inline void mem_cgroup_enter_user_fault(void)
{
- WARN_ON(current->memcg_may_oom);
- current->memcg_may_oom = 1;
+ WARN_ON(current->in_user_fault);
+ current->in_user_fault = 1;
}
-static inline void mem_cgroup_oom_disable(void)
+static inline void mem_cgroup_exit_user_fault(void)
{
- WARN_ON(!current->memcg_may_oom);
- current->memcg_may_oom = 0;
+ WARN_ON(!current->in_user_fault);
+ current->in_user_fault = 0;
}
static inline bool task_in_memcg_oom(struct task_struct *p)
@@ -948,11 +948,11 @@ static inline void mem_cgroup_handle_ove
{
}
-static inline void mem_cgroup_oom_enable(void)
+static inline void mem_cgroup_enter_user_fault(void)
{
}
-static inline void mem_cgroup_oom_disable(void)
+static inline void mem_cgroup_exit_user_fault(void)
{
}
diff -puN include/linux/sched.h~memcg-oom-move-out_of_memory-back-to-the-charge-path include/linux/sched.h
--- a/include/linux/sched.h~memcg-oom-move-out_of_memory-back-to-the-charge-path
+++ a/include/linux/sched.h
@@ -722,7 +722,7 @@ struct task_struct {
unsigned restore_sigmask:1;
#endif
#ifdef CONFIG_MEMCG
- unsigned memcg_may_oom:1;
+ unsigned in_user_fault:1;
#ifndef CONFIG_SLOB
unsigned memcg_kmem_skip_account:1;
#endif
diff -puN mm/memcontrol.c~memcg-oom-move-out_of_memory-back-to-the-charge-path mm/memcontrol.c
--- a/mm/memcontrol.c~memcg-oom-move-out_of_memory-back-to-the-charge-path
+++ a/mm/memcontrol.c
@@ -1534,28 +1534,53 @@ static void memcg_oom_recover(struct mem
__wake_up(&memcg_oom_waitq, TASK_NORMAL, 0, memcg);
}
-static void mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int order)
+enum oom_status {
+ OOM_SUCCESS,
+ OOM_FAILED,
+ OOM_ASYNC,
+ OOM_SKIPPED
+};
+
+static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int order)
{
- if (!current->memcg_may_oom || order > PAGE_ALLOC_COSTLY_ORDER)
- return;
+ if (order > PAGE_ALLOC_COSTLY_ORDER)
+ return OOM_SKIPPED;
+
/*
* We are in the middle of the charge context here, so we
* don't want to block when potentially sitting on a callstack
* that holds all kinds of filesystem and mm locks.
*
- * Also, the caller may handle a failed allocation gracefully
- * (like optional page cache readahead) and so an OOM killer
- * invocation might not even be necessary.
+ * cgroup1 allows disabling the OOM killer and waiting for outside
+ * handling until the charge can succeed; remember the context and put
+ * the task to sleep at the end of the page fault when all locks are
+ * released.
*
- * That's why we don't do anything here except remember the
- * OOM context and then deal with it at the end of the page
- * fault when the stack is unwound, the locks are released,
- * and when we know whether the fault was overall successful.
+ * On the other hand, in-kernel OOM killer allows for an async victim
+ * memory reclaim (oom_reaper) and that means that we are not solely
+ * relying on the oom victim to make a forward progress and we can
+ * invoke the oom killer here.
+ *
+ * Please note that mem_cgroup_out_of_memory might fail to find a
+ * victim and then we have to bail out from the charge path.
*/
- css_get(&memcg->css);
- current->memcg_in_oom = memcg;
- current->memcg_oom_gfp_mask = mask;
- current->memcg_oom_order = order;
+ if (memcg->oom_kill_disable) {
+ if (!current->in_user_fault)
+ return OOM_SKIPPED;
+ css_get(&memcg->css);
+ current->memcg_in_oom = memcg;
+ current->memcg_oom_gfp_mask = mask;
+ current->memcg_oom_order = order;
+
+ return OOM_ASYNC;
+ }
+
+ if (mem_cgroup_out_of_memory(memcg, mask, order))
+ return OOM_SUCCESS;
+
+ WARN(1,"Memory cgroup charge failed because of no reclaimable memory! "
+ "This looks like a misconfiguration or a kernel bug.");
+ return OOM_FAILED;
}
/**
@@ -1950,6 +1975,8 @@ static int try_charge(struct mem_cgroup
unsigned long nr_reclaimed;
bool may_swap = true;
bool drained = false;
+ bool oomed = false;
+ enum oom_status oom_status;
if (mem_cgroup_is_root(memcg))
return 0;
@@ -2037,6 +2064,9 @@ retry:
if (nr_retries--)
goto retry;
+ if (gfp_mask & __GFP_RETRY_MAYFAIL && oomed)
+ goto nomem;
+
if (gfp_mask & __GFP_NOFAIL)
goto force;
@@ -2045,8 +2075,23 @@ retry:
memcg_memory_event(mem_over_limit, MEMCG_OOM);
- mem_cgroup_oom(mem_over_limit, gfp_mask,
+ /*
+ * keep retrying as long as the memcg oom killer is able to make
+ * a forward progress or bypass the charge if the oom killer
+ * couldn't make any progress.
+ */
+ oom_status = mem_cgroup_oom(mem_over_limit, gfp_mask,
get_order(nr_pages * PAGE_SIZE));
+ switch (oom_status) {
+ case OOM_SUCCESS:
+ nr_retries = MEM_CGROUP_RECLAIM_RETRIES;
+ oomed = true;
+ goto retry;
+ case OOM_FAILED:
+ goto force;
+ default:
+ goto nomem;
+ }
nomem:
if (!(gfp_mask & __GFP_NOFAIL))
return -ENOMEM;
diff -puN mm/memory.c~memcg-oom-move-out_of_memory-back-to-the-charge-path mm/memory.c
--- a/mm/memory.c~memcg-oom-move-out_of_memory-back-to-the-charge-path
+++ a/mm/memory.c
@@ -4131,7 +4131,7 @@ int handle_mm_fault(struct vm_area_struc
* space. Kernel faults are handled more gracefully.
*/
if (flags & FAULT_FLAG_USER)
- mem_cgroup_oom_enable();
+ mem_cgroup_enter_user_fault();
if (unlikely(is_vm_hugetlb_page(vma)))
ret = hugetlb_fault(vma->vm_mm, vma, address, flags);
@@ -4139,7 +4139,7 @@ int handle_mm_fault(struct vm_area_struc
ret = __handle_mm_fault(vma, address, flags);
if (flags & FAULT_FLAG_USER) {
- mem_cgroup_oom_disable();
+ mem_cgroup_exit_user_fault();
/*
* The task may have entered a memcg OOM situation but
* if the allocation error was handled gracefully (no
_
Patches currently in -mm which might be from mhocko@xxxxxxxx are
mm-drop-vm_bug_on-from-__get_free_pages.patch
memcg-oom-move-out_of_memory-back-to-the-charge-path.patch
mm-oom-docs-describe-the-cgroup-aware-oom-killer-fix-2.patch
--
To unsubscribe from this list: send the line "unsubscribe mm-commits" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
