<quote who="David Cruz">
> Other optimizations people are testing out there.
>
> - use "nohz=off" in the kernel loading line y menu.lst
> - Disable Cgroups completely. Using cgclear, and turning off cgred
> cg-config daemons.
>
> And from a Personal point of view, we've always tried to use MySQL in
> a different server from JBoss.
> 99% of the times is far better for performance and tuning.
I am still having problems. Running mysql and JBoss on different VMs is
significantly slower, but that appears to be a pure application issue
relating to the network overhead in a huge amount of queries. This is of
course not an issue for this mailing list.
However, I got a performance degradation of running the same test with
MySQL and JBoss colocated and increasing the CPU count from 4 to 8. Then I
did various tricks to improve performance. Pinning the processes did not
have a significant effect. What appeared to have an effect was to remove
memory ballooning.
But now I get inconsistent results, varying from 21 seconds to 27 seconds.
The 21 figure is acceptable but the 27 figure is not. The tests are being
done on new hardware (see previous posts), with basically the only thing
running the virtual machine.
Summary of configuration:
- host and guest: centos 6.3, transparent hugepages
- vm: cpu mode is host-passthrough, pinning to the cores of one processor,
removed tablet, sound, and USB devices
- host: hyperthreading switched off in BIOS
Do you have any idea what this could be? I expect it is somehow NUMA
related, but how would I troubleshoot this? Ideally I would like to make
sure the entire VM runs on one CPU and allocates memory from that CPU and
never moves (or both CPU and memory move together). I saw some
presentations on the internet about NUMA work being done for linux. Do you
have any suggestions?
My domain.xml is given below:
<domain type='kvm' id='37'>
<name>master-data05-v50</name>
<uuid>79ddd84d-937e-357b-8e57-c7f487dc3464</uuid>
<memory unit='KiB'>8388608</memory>
<currentMemory unit='KiB'>8388608</currentMemory>
<vcpu placement='static'>8</vcpu>
<cputune>
<vcpupin vcpu='0' cpuset='0'/>
<vcpupin vcpu='1' cpuset='2'/>
<vcpupin vcpu='2' cpuset='4'/>
<vcpupin vcpu='3' cpuset='6'/>
<vcpupin vcpu='4' cpuset='8'/>
<vcpupin vcpu='5' cpuset='10'/>
<vcpupin vcpu='6' cpuset='12'/>
<vcpupin vcpu='7' cpuset='14'/>
</cputune>
<os>
<type arch='x86_64' machine='rhel6.3.0'>hvm</type>
<boot dev='cdrom'/>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic/>
</features>
<cpu mode='host-passthrough'>
</cpu>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
<disk type='file' device='cdrom'>
<driver name='qemu' type='raw'/>
<target dev='hdc' bus='ide'/>
<readonly/>
<alias name='ide0-1-0'/>
<address type='drive' controller='0' bus='1' target='0' unit='0'/>
</disk>
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none' io='native'/>
<source dev='/dev/raid5/v50disk1'/>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05'
function='0x0'/>
</disk>
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none' io='native'/>
<source dev='/dev/vg_system/v50disk2'/>
<target dev='vdb' bus='virtio'/>
<alias name='virtio-disk1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08'
function='0x0'/>
</disk>
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none' io='native'/>
<source dev='/dev/raid5/v50disk3'/>
<target dev='vdc' bus='virtio'/>
<alias name='virtio-disk2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09'
function='0x0'/>
</disk>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/var/mydata/images/configdisks/v50/configdisk.img'/>
<target dev='vdz' bus='virtio'/>
<alias name='virtio-disk25'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07'
function='0x0'/>
</disk>
<controller type='usb' index='0'>
<alias name='usb0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x2'/>
</controller>
<controller type='ide' index='0'>
<alias name='ide0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x1'/>
</controller>
<interface type='bridge'>
<mac address='52:54:00:00:01:50'/>
<source bridge='br0'/>
<target dev='vnet0'/>
<model type='virtio'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03'
function='0x0'/>
</interface>
<serial type='pty'>
<source path='/dev/pts/1'/>
<target port='0'/>
<alias name='serial0'/>
</serial>
<console type='pty' tty='/dev/pts/1'>
<source path='/dev/pts/1'/>
<target type='serial' port='0'/>
<alias name='serial0'/>
</console>
<input type='mouse' bus='ps2'/>
<graphics type='vnc' port='5900' autoport='yes'/>
<video>
<model type='cirrus' vram='9216' heads='1'/>
<alias name='video0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x0'/>
</video>
</devices>
<seclabel type='none'/>
</domain>
The output of virsh capabilities:
<capabilities>
<host>
<uuid>44454c4c-4600-1031-8043-b9c04f47354a</uuid>
<cpu>
<arch>x86_64</arch>
<model>SandyBridge</model>
<vendor>Intel</vendor>
<topology sockets='1' cores='8' threads='1'/>
<feature name='pdpe1gb'/>
<feature name='osxsave'/>
<feature name='tsc-deadline'/>
<feature name='dca'/>
<feature name='pdcm'/>
<feature name='xtpr'/>
<feature name='tm2'/>
<feature name='est'/>
<feature name='smx'/>
<feature name='vmx'/>
<feature name='ds_cpl'/>
<feature name='monitor'/>
<feature name='dtes64'/>
<feature name='pbe'/>
<feature name='tm'/>
<feature name='ht'/>
<feature name='ss'/>
<feature name='acpi'/>
<feature name='ds'/>
<feature name='vme'/>
</cpu>
<power_management>
<suspend_disk/>
</power_management>
<migration_features>
<live/>
<uri_transports>
<uri_transport>tcp</uri_transport>
</uri_transports>
</migration_features>
<topology>
<cells num='2'>
<cell id='0'>
<cpus num='8'>
<cpu id='0'/>
<cpu id='2'/>
<cpu id='4'/>
<cpu id='6'/>
<cpu id='8'/>
<cpu id='10'/>
<cpu id='12'/>
<cpu id='14'/>
</cpus>
</cell>
<cell id='1'>
<cpus num='8'>
<cpu id='1'/>
<cpu id='3'/>
<cpu id='5'/>
<cpu id='7'/>
<cpu id='9'/>
<cpu id='11'/>
<cpu id='13'/>
<cpu id='15'/>
</cpus>
</cell>
</cells>
</topology>
</host>
<guest>
<os_type>hvm</os_type>
<arch name='i686'>
<wordsize>32</wordsize>
<emulator>/usr/libexec/qemu-kvm</emulator>
<machine>rhel6.3.0</machine>
<machine canonical='rhel6.3.0'>pc</machine>
<machine>rhel6.2.0</machine>
<machine>rhel6.1.0</machine>
<machine>rhel6.0.0</machine>
<machine>rhel5.5.0</machine>
<machine>rhel5.4.4</machine>
<machine>rhel5.4.0</machine>
<domain type='qemu'>
</domain>
<domain type='kvm'>
<emulator>/usr/libexec/qemu-kvm</emulator>
</domain>
</arch>
<features>
<cpuselection/>
<deviceboot/>
<pae/>
<nonpae/>
<acpi default='on' toggle='yes'/>
<apic default='on' toggle='no'/>
</features>
</guest>
<guest>
<os_type>hvm</os_type>
<arch name='x86_64'>
<wordsize>64</wordsize>
<emulator>/usr/libexec/qemu-kvm</emulator>
<machine>rhel6.3.0</machine>
<machine canonical='rhel6.3.0'>pc</machine>
<machine>rhel6.2.0</machine>
<machine>rhel6.1.0</machine>
<machine>rhel6.0.0</machine>
<machine>rhel5.5.0</machine>
<machine>rhel5.4.4</machine>
<machine>rhel5.4.0</machine>
<domain type='qemu'>
</domain>
<domain type='kvm'>
<emulator>/usr/libexec/qemu-kvm</emulator>
</domain>
</arch>
<features>
<cpuselection/>
<deviceboot/>
<acpi default='on' toggle='yes'/>
<apic default='on' toggle='no'/>
</features>
</guest>
</capabilities>
The output of virsh vcpuinfo master-data05-v50:
VCPU: 0
CPU: 0
State: running
CPU time: 119.4s
CPU Affinity: y---------------
VCPU: 1
CPU: 2
State: running
CPU time: 78.9s
CPU Affinity: --y-------------
VCPU: 2
CPU: 4
State: running
CPU time: 74.4s
CPU Affinity: ----y-----------
VCPU: 3
CPU: 6
State: running
CPU time: 61.6s
CPU Affinity: ------y---------
VCPU: 4
CPU: 8
State: running
CPU time: 91.4s
CPU Affinity: --------y-------
VCPU: 5
CPU: 10
State: running
CPU time: 67.4s
CPU Affinity: ----------y-----
VCPU: 6
CPU: 12
State: running
CPU time: 52.8s
CPU Affinity: ------------y---
VCPU: 7
CPU: 14
State: running
CPU time: 66.8s
CPU Affinity: --------------y-
I also checked x2apic using 'grep -i x2apic /var/log/messages*'. this
shows x2apic is not working on the host but it is working on the guest.
Should I enable it on the host as well as suggested by the output?
On the host I get:
/var/log/messages-20130210:Feb 8 08:51:00 localhost kernel: Your BIOS has
requested that x2apic be disabled.
/var/log/messages-20130210:Feb 8 08:51:00 localhost kernel: Use
'intremap=no_x2apic_optout' to override BIOS request.
/var/log/messages-20130210:Feb 8 08:51:00 localhost kernel: x2apic not
enabled, IRQ remapping is in xapic mode
On the guest I get:
/var/log/messages-20130213:Feb 13 02:00:19 localhost kernel: Setting APIC
routing to physical x2apic
/var/log/messages-20130213:Feb 13 02:08:11 localhost kernel: Enabling x2apic
/var/log/messages-20130213:Feb 13 02:08:11 localhost kernel: Enabled x2apic
/var/log/messages-20130213:Feb 13 02:08:11 localhost kernel: Setting APIC
routing to physical x2apic
>
> David
>
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
