KVM and the OOM-Killer

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



This is *NOT* a KVM issue, but may be worth adding into the FAQ...


We have a KVM host with 48Gb of RAM and run about 20 KVM clients on it. After some time - different time depending on the kernel version - the VM host kernel will start OOM-Killing the VM clients, even when there is lots of free RAM (>10Gb) and free SWAP (>34Gb).

This seems to be caused by the kernel running out of LOWMEM (memory below 1Gb) - because of the large amount of RAM a lot of LOWMEM (~400Mb) is used by the memory map (32 bytes per 4Kb page), add in the kernel itself and that leaves "only" about 460Mb of LOWMEM for kernel alloc.

This may not have been a problem, except Linux may also put cache blocks and user processes in LOWMEM - it seems this can then lead to a LOWMEM exhaust situation which triggers OOM-Killing even when there is LOADS of SWAP and HIGHMEM free.

Sadly, killing userland processes is not a good way to try and free LOWMEM, so what happens is a killing spree where by every process on the VM host gets killed (inc all the VMs, sysklogd, klogd, sshd, udevd etc).

This is very bad in 2.6.32.6, quite bad in 2.6.32.9, better (but still bad in) 2.6.31.12 - currently testing 2.6.33.3

See https://bugzilla.kernel.org/show_bug.cgi?id=15058


General advice seems to be, if you have more than 16Gb RAM then you should run the VM host 64bit.

We didn't see this issue on a server with 32Gb running the same set of VMs.



James
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]
  Powered by Linux