Re: kernel memory accounting

[Jonathan Billings <billings@xxxxxxxxxx>]

On Mar 10, 2017, at 3:51 PM, Wensheng Deng <wd35@xxxxxxx> wrote:
> 
> I have 3.10 kernel. I am running some data processing job, need to first
> copy big (>5 GB) input files. The jobs were killed, because the system
> thought I used 5 GB memory from the file copying.

If you’re using ‘cp’ you probably aren’t using 5G of RAM.  That’s not how ‘cp’ works.  Actual errors might be helpful here.

If you are running batch processing and don’t want the OOM Killer to ever get involved, the cgroup memory accounting features actually let you turn it off with memory.oom_control=1 in the cgroup.  You can also turn off the heuristic overcommit memory manager (see https://www.kernel.org/doc/Documentation/vm/overcommit-accounting for details) but I suggest figuring out your problem with copying files first.  

--
Jonathan Billings <billings@xxxxxxxxxx>


_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos