Re: PROBLEM: Memory leaking when running kubernetes cronjobs

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, Sep 20, 2018 at 08:23:06AM +0000, Daniel McGinnes wrote:
> Hi Roman,
> 
> unfortunately Kubernetes seems to be using version 1 cgroups, so I can't 
> see that stat - I'll investigate if there's a way to get Kube to use V2 so 
> we can check this..

Hi Daniel!

Yeah, it might be not so easy, AFAIK.
Alternatively, you can expose this cgroup v2 data in v1 interface
using an off-stream patch, just for debugging. Should be pretty
straightforward; I can help with it, if necessary.

> 
> There wasn't memory pressure, I just run it in a pretty controlled way 
> when running the test - so initially it sounds like what I saw was 
> expected. I then ran stress --vm 16 --vm-bytes 2147483648 which did create 
> some memory pressure and I saw oom killer getting invoked - it seemed 
> pretty similar behaviour to before where only a small amount of the "lost" 
> memory was reclaimed... Maybe I was being too severe with stress and the 
> memory would be reclaimed at a slower rate under more reasonable memory 
> pressure?

So, did you run the stress -vm after the main workload or in parallel?
Can you, please, try to create some ambient memory pressure?
Does echo 3 > /proc/sys/vm/drop_caches help to reclaim the memory?

Thanks!

Roman




[Index of Archives]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]     [Monitors]

  Powered by Linux