On Tue, Jun 21, 2011 at 7:51 AM, Lutz Vieweg <lvml@xxxxxx> wrote: > Dear Memory Ressource Controller maintainers, > > by using per-user control groups with a limit on memory (and swap) I am > trying to secure a shared development server against memory exhaustion > by any one single user - as it happened before when somebody imprudently > issued "make -j" (which has the infamous habit to spawn an unlimited > number of processes) on a large software project with many source files. > > The memory limitation using control groups works just fine when > only a few processes sum up to a usage that exceeds the limits - the > processes are OOM-killed, then, and the others users are unaffected. > > But the original cause, a "make -j" on many source files, leads to > the following ugly symptom: > > - make starts numerous (~ 100 < x < 200) gcc processes > > - some of those gcc processes get OOM-killed quickly, then > a few more are killed, but with increasing pauses in between > > - then after a few seconds, no more gcc processes are killed, but > the "make" process and its childs do not show any progress anymore > > - at this time, top indicates 100% "system" CPU usage, mostly by > "[kworker/*]" threads (one per CPU). But processes from other > users, that only require CPU, proceed to run. The following patch might not be the root-cause of livelock, but should reduce the [kworker/*] in your case. == >From d1372da4d3c6f8051b5b1cf7b5e8b45a8094b388 Mon Sep 17 00:00:00 2001 From: KAMEZAWA Hiroyuki <kamezawa.hiroyu@xxxxxxxxxxxxxx> Date: Wed, 8 Jun 2011 13:51:11 +0900 Subject: [BUGFIX][PATCH] memcg: fix behavior of per cpu charge cache draining. Can you give a try? I don't know which kernel you are using in case you don't have this patched yet. --Ying > > - but also at this time, if any other user (who has not exhausted > his memory limits) tries to access any file (at least on /tmp/, > as e.g. gcc does), even a simple "ls /tmp/", this operation > waits forever. (But "iostat" does not indicate any I/O activity.) > > - as soon as you press "CTRL-C" to abort the "make -j", everything > goes back to normal, quickly - also the other users' processes proceed. > > > To reproduce the problem, the attached "Makefile" to a directory > on a filesystem with at least 70MB free space, then > > mount -o memory none /cgroup > mkdir /cgroup/test > echo 64M >/cgroup/test/memory.limit_in_bytes > echo 64M >/cgroup/test/memory.memsw.limit_in_bytes > > cd /somewhere/with/70mb/free > echo $$ >/cgroup/test/tasks > make sources > make -j compile > > Notice that "make sources" will create 200 bogus "*.c" files from > /dev/urandom to make sure that "gcc" will use up some memory. > > The "make -j compile" reliably reproduces the above mentioned syndrome, > here. > > Please notice that the livelock does happen only with a significant > number of parallel compiler runs - it did e.g. not happen with > only 100 for me, and it also did not happen when I started "make" > with "strace" - so timing seems to be an issue, here. > > Thanks for any hints towards a solution of this issue in advance! > > Regards, > > Lutz Vieweg > -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: <a href