Hi,
I just had a chance to play with KVM on Ubuntu 10.04 LTS on some new HP
360 g6 with Nehalem processors. I have a feeling that KVM and NUMA on
these machines do not play well together.
Doing some benchmarks I got bizarre numbers. Sometimes the VMs were
performing fine and some times the performance was very bad! Apparently
KVM does not recognize the NUMA-architecture and places memory and
process randomly and therefore often on different numa cells.
First a couple of specs of the machine:
Two Nehalem sockets with E5520, Hyperthreading turned off, 4 cores per
socket, all in all 8 processors.
# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 26
model name : Intel(R) Xeon(R) CPU E5520 @ 2.27GHz
stepping : 5
...
Linux recognizes the NUMA-architecture:
# numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 2 4 6
node 0 size: 12277 MB
node 0 free: 9183 MB
node 1 cpus: 1 3 5 7
node 1 size: 12287 MB
node 1 free: 8533 MB
node distances:
node 0 1
0: 10 20
1: 20 10
So I have got two cells with 4 cores each.
Virsh does not recognize the topology:
# virsh capabilities
<capabilities>
<host>
<cpu>
<arch>x86_64</arch>
<model>core2duo</model>
<topology sockets='2' cores='4' threads='1'/>
<feature name='lahf_lm'/>
..
I guess this is the fact, because QEMU does not recognize the
NUMA-Architecture (QEMU-Monitor):
(qemu) info numa
0 nodes
This is done on an Ubuntu 10.04 LTS:
Linux lxkvm01 2.6.32-23-server #37-Ubuntu SMP Fri Jun 11 09:11:11 UTC
2010 x86_64 GNU/Linux
The package used was:
qemu-kvm 0.12.3+noroms-0ubuntu9.2
When doing benchmarks numastat shows a lot of misses when the
performance is bad.
# numastat
node0 node1
numa_hit 15527158 17015505
numa_miss 7032982 3512950
numa_foreign 3512950 7032982
interleave_hit 8078 8264
local_node 15525187 17006655
other_node 7034953 3521800
So apparently KVM does not utilize the NUMA-architecture. Did I do
something wrong. Is KVM missing a patch? Do I need to activate something
in KVM to recognize the NUMA-Architecture?
I tried the newest Kernel Module 2.6.32.15 and Qemu-kvm 0.12.4 without a
change.
Any hints?
Kind regards,
Ralf
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
