On 2016-05-17 12:16, Sebastian Frias wrote:
Hi Michal,
On 05/17/2016 10:57 AM, Michal Hocko wrote:
On Tue 17-05-16 10:24:20, Sebastian Frias wrote:
[...]
Also, under what conditions would copy-on-write fail?
When you have no memory or swap pages free and you touch a COW page that
is currently shared. At that point there is no resource to back to the
copy so something must die - either the process doing the copy or
something else.
Exactly, and why does "killing something else" makes more sense (or
was chosen over) "killing the process doing the copy"?
Because that "something else" is usually a memory hog and so chances are
that the out of memory situation will get resolved. If you kill "process
doing the copy" then you might end up just not getting any memory back
because that might be a little forked process which doesn't own all that
much memory on its own. That would leave you in the oom situation for a
long time until somebody actually sitting on some memory happens to ask
for CoW... See the difference?
I see the difference, your answer seems a bit like the one from Austin, basically:
- killing a process is a sort of kernel protection attempting to deal "automatically" with some situation, like deciding what is a 'memory hog', or what is 'in infinite loop', "usually" in a correct way.
It seems there's people who think its better to avoid having to take such decisions and/or they should be decided by the user, because "usually" != "always".
FWIW, it's really easy to see what's using a lot of memory, it's
impossible to tell if something is stuck in an infinite loop without
looking deep into the process state and possibly even at the source code
(and even then it can be almost impossible to be certain). This is why
we have a OOM-Killer, and not a infinite-loop-killer.
Again I reiterate, if a system is properly provisioned (that is, if you
have put in enough RAM and possibly swap space to do what you want to
use it for), the only reason the OOM-killer should be invoked is due to
a bug. The non-default overcommit options still have the same issues
they just change how and when they happen (overcommit=never will fire
sooner, overcommit=always will fire later), and also can impact memory
allocation performance (I have numbers somewhere that I can't find right
now that demonstrated that overcommit=never gave more deterministic and
(on average) marginally better malloc() performance, and simple logic
would suggest that overcommit=always would make malloc() perform better
too).
And people who see that as a nice thing but complex thing to do.
In this thread we've tried to explain why this heuristic (and/or OOM-killer) is/was needed and/or its history, which has been very enlightening by the way.
From reading Documentation/cgroup-v1/memory.txt (and from a few replies here talking about cgroups), it looks like the OOM-killer is still being actively discussed, well, there's also "cgroup-v2".
My understanding is that cgroup's memory control will pause processes in a given cgroup until the OOM situation is solved for that cgroup, right?
If that is right, it means that there is indeed a way to deal with an OOM situation (stack expansion, COW failure, 'memory hog', etc.) in a better way than the OOM-killer, right?
In which case, do you guys know if there is a way to make the whole system behave as if it was inside a cgroup? (*)
No, not with the process freeze behavior, because getting the group
running again requires input from an external part of the system, which
by definition doesn't exist if the group is the entire system; and,
because our GUI isn't built into the kernel, we can't pause things and
pop up a little dialog asking the user what to do to resolve the issue.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxx. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>