Re: cgroups: cannot write on file memory.oom_control

[Hedi Boufaied <hedi.boufaied@xxxxxxxxxxx>]

Hi Rami,

I have looked at the example code in
your slides and wrote something similar to handle the OoM condition and
it seems to work fine. Thank you for that.

Since it would be even better for me
to avoid playing with OoM, I explored another option for monitoring memory
usage: keeping the OoM killer enabled and using the "memory threshold"
feature (see below) to get notified when a group's memory usage exceeds
the limit.

First, my C code for monitoring the
memory usage looked like this (running as a separate process): 

// Assuming that initial memory usage
is below the threshold 
 while(1)
    {
      if (read(efd, &counter,
sizeof(uint64_t)) != sizeof(uint64_t)) die("read eventfd");
      printf("Event
received: memory usage ABOVE limit\n");
      if (read(efd, &counter,
sizeof(uint64_t)) != sizeof(uint64_t)) die("read eventfd");
      printf("Event
received: memory usage BELOW limit\n");
    }
  
Then I thought: what happens if mem
usage crosses the threshold more than once between 2 consecutive calls
to read() ? (a scenario that cannot be formally excluded).
I realised that in this situation the
"counter" value returned by read() would contain the number of
occurrences crossing events. 
So for example if, between 2 calls to
read(), the memory usage goes beyond then below the threshold, the value
of "counter" after the 2nd read will be 2.

I ended up using this counter behaviour
to determine if the current usage is below or beyond the limit:

counter = 0;
above_mem_limit = 0;

while(1)
  {
    if (read(efd, &counter,
sizeof(uint64_t)) != sizeof(uint64_t)) die("read eventfd");
    if(counter%2) above_mem_limit
= !above_mem_limit;
    printf("Event received!
(counter=%d), above_mem_limit=%d\n", counter, above_mem_limit);
 }

My question: I have never seen this
"event counter" behaviour documented anywhere. The cgroups docs
and forums do not say anything about the 64bit value returned by the read()
call.  Do you think one can safely rely on this to count the number
of crossing events ?

Thanks and Regards,
Hedi

----

From https://www.kernel.org/doc/Documentation/cgroups/memory.txt:

9. Memory thresholds

Memory cgroup implements memory thresholds using the cgroups notification
API (see cgroups.txt). It allows to register multiple memory and memsw
thresholds and gets notifications when it crosses.

To register a threshold, an application must:
- create an eventfd using eventfd(2);
- open memory.usage_in_bytes or memory.memsw.usage_in_bytes;
- write string like "<event_fd> <fd of memory.usage_in_bytes>
<threshold>" to
  cgroup.event_control.

Application will be notified through eventfd when memory usage crosses
threshold in any direction.

It's applicable for root and non-root cgroup.

----
Best Regard,
Hedi



From:      
 Hedi Boufaied/NCE/AMADEUS
To:      
 Rami Rosen <roszenrami@xxxxxxxxx>@AMAEXTMAIL_MUC,

Cc:      
 kernelnewbies <kernelnewbies@xxxxxxxxxxxxxxxxx>
Date:      
 22/04/2014 13:12
Subject:    
   Re: cgroups:
cannot write on file memory.oom_control

---

Hi Rami,

Thank you for the advice. Yes, I will
try using the event handler.

Best Regards,
Hedi




From:      
 Rami Rosen <roszenrami@xxxxxxxxx>
To:      
 Hedi Boufaied <hedi.boufaied@xxxxxxxxxxx>,

Cc:      
 kernelnewbies <kernelnewbies@xxxxxxxxxxxxxxxxx>
Date:      
 18/04/2014 22:38
Subject:    
   Re: cgroups:
cannot write on file memory.oom_control
Sent by:    
   kernelnewbies-bounces@xxxxxxxxxxxxxxxxx

---

Hi Hedi,
Well, you can use disabling oom in conjunction  with event handler.
Thus, you can free memory by changing memory setting of other cgroups when
an oom occurs in a specific cgroup, and avoid having an oops in that cgroup.
You can see an example of using event handler in conjunction with disabling
the oom killer in 
"Namespaces and cgroups in linux", in slides 101/102, see:
http://ramirose.wix.com/ramirosen

A shameless plug - I wrote this presentation of 121 slides about namespaces and cgroups.

Regards,
Rami Rosen

בתאריך 17 באפר 2014 11:24, "Hedi Boufaied" <hedi.boufaied@xxxxxxxxxxx> כתב:
Hi Rami,

Thanks a lot for your sequence. I followed it and it does work.
I now see my process paused in sleep state instead of getting killed when the cgroup reaches OoO.

This being said:
* I do not really understand why the use_hierarchy option needs to be disabled in order to allow disabling of the oom killer...
* I'm not sure that disabling the OoO killer is safe/reliable enough: in some of the trials I made, I saw other processes (xterms) not part of the OoO cgroup getting completely stuck and I had to reboot my machine...

Best Regards,
Hedi



From:        Rami Rosen <roszenrami@xxxxxxxxx>
To:        Hedi Boufaied <hedi.boufaied@xxxxxxxxxxx>,
Cc:        kernelnewbies <kernelnewbies@xxxxxxxxxxxxxxxxx>
Date:        15/04/2014 15:39
Subject:        Re: cgroups: cannot write on file memory.oom_control

---

Hi, Hedi,

This sequence works for me:

echo 0 >  /sys/fs/cgroup/memory/memory.use_hierarchy

mkdir  /sys/fs/cgroup/memory/0

Then, the following two commands changes the value of oom_kill_disable:

echo 1 >   /sys/fs/cgroup/memory/0/memory.oom_control

echo 0 >   /sys/fs/cgroup/memory/0/memory.oom_control


For more info, please look for the text about enabling/disabling
use_hierarchy in section 6,
http://lxr.free-electrons.com/source/Documentation/cgroups/memory.txt.

Regards,
Rami Rosen
http://ramirose.wix.com/ramirosen



On Mon, Apr 14, 2014 at 12:22 PM, Hedi Boufaied
<hedi.boufaied@xxxxxxxxxxx>
wrote:
> Hi everyone,
>
> I am working with cgroups on OpenSuse (over VirtualBox) and I cannot
disable
> the OOM killer by writing to the file memory.oom_control:
>
>> echo 1 > memory.oom_control
>> -bash: echo: write error: Invalid argument
>
> I can change the memory limit and several other settings by writing
to the
> appropriate files (like memory.limit_in_bytes) but I could never write
to
> file memory.oom_control although I am root.
>
> I saw someone posted a similar issue a few days ago but there was
no
> reply...
>
> I have copied below the sequence of command I am using. Any idea what
could
> be the issue ?
>
> Thanks in advance for your help!
>
> Hedi
>
>
> ----
>
> /sys/fs> su - root
>
> /sys/fs> cd cgroup
>
> /sys/fs/cgroup> cgcreate -g memory:/mygroup
>
> /sys/fs/cgroup> cd memory/mygroup
>
> /sys/fs/cgroup/memory/mygroup> echo 32M > memory.limit_in_bytes
>
> /sys/fs/cgroup/memory/mygroup> cat memory.limit_in_bytes
> 33554432
>
> /sys/fs/cgroup/memory/mygroup> echo 1 > memory.oom_control
> -bash: echo: write error: Invalid argument
>
> /sys/fs/cgroup/memory/mygroup> cat memory.oom_control
> oom_kill_disable 0
> under_oom 0
>
> ---
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies@xxxxxxxxxxxxxxxxx
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@xxxxxxxxxxxxxxxxx
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@xxxxxxxxxxxxxxxxx
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies