Re: Live Migration and LibVirt CPU Mode

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Dear Oisin,
I would recommend you do the offline migration(Export/Import) instead due to CPU spec are different, refer link--> "https://bugs.launchpad.net/nova/+bug/1082414" and "https://review.openstack.org/#/c/161944/2". This save a lot more time instead of bug digging.

Hope that helps.
Xlord

On Thu, Jul 6, 2017 at 9:29 AM, Oisin O'Malley <oisin.omalley@xxxxxxxxxxxxx> wrote:
Hi All,

First time mailing here, I hope someone can help. We’re running a OpenStack Newton environment on top of CentOS-7.3, LibVirt and Qemu-KVM-EV. We are encountered an issue live migrating a VM between 2 hosts with different CPUs and LibVirt throws the following error:

libvirtError: unsupported configuration: guest and host CPU are not compatible: Host CPU does not provide required features: fma, x2apic, movbe, tsc-deadline, xsave, osxsave, avx, f16c, rdrand, fsgsbase, tsc_adjust, bmi1, hle, avx2, smep, bmi2, erms, invpcid, rtm, rdseed, adx, smap, xsaveopt, abm, 3dnowprefetch; try using 'Broadwell-noTSX' CPU model.


LibVirt appears to be comparing the source and destination host CPUs, instead of guest VM and destination host CPUs. The VM is configured with a common set CPU features that are compatible with both (see below) and can run on both hosts without issue. The hosts have Westmere and Broadwell CPUs (See host CPU capabilities below). The VM can be booted on the Westmere host, successfully live migrated to the Broadwell host, but then throws the above error when we attempt to live migrating back to the Westmere host.

Due to how our Westmere hosts were configured in OpenStack Nova, the VMs CPU is configured with "cpu mode='host-model'", which fail to migrate. BUT if the VMs CPU is configured with "cpu mode='custom' match='exact'" it can be successfully migrate both ways. It appears that when a VMs CPU is configured with "cpu mode='host-model'" the guests CPU is ignored and the host CPUs are compared instead. Which I believe is incorrect behaviour, is this a known issue and/or have any workarounds.

The VMs will be configure to use "cpu mode='custom'" with a Westmere model on their next reboot, but rebooting all VMs prior to migration is something we really want to avoid.

There has already been some discussion on this issue on the OpenStack mailing list: http://lists.openstack.org/pipermail/openstack/2017-July/thread.html#45139

The following package versions are being used;
libvirt-daemon-kvm.x86_642.0.0-10.el7_3.4
qemu-kvm-ev.x86_6410:2.6.0-28.el7_3.3.1
kernel.x86_643.10.0-514.16.1.el7


The Qemu-KVM processes CPU parameters of the running VMs;

-cpu Westmere,+vme,+ds,+acpi,+ss,+ht,+tm,+pbe,+pclmuldq,+dtes64,+monitor,+ds_cpl,+vmx,+smx,+est,+tm2,+xtpr,+pdcm,+pcid,+dca,+arat,+pdpe1gb,+rdtscp


Guest CPU Configuration that fails to migrate;

<cpu mode='host-model'>
    <model fallback='allow'>Westmere</model>
    <vendor>Intel</vendor>
    <topology sockets='1' cores='1' threads='1'/>
    <feature policy='require' name='vme'/>
    <feature policy='require' name='ds'/>
    <feature policy='require' name='acpi'/>
    <feature policy='require' name='ss'/>
    <feature policy='require' name='ht'/>
    <feature policy='require' name='tm'/>
    <feature policy='require' name='pbe'/>
    <feature policy='require' name='pclmuldq'/>
    <feature policy='require' name='dtes64'/>
    <feature policy='require' name='monitor'/>
    <feature policy='require' name='ds_cpl'/>
    <feature policy='require' name='vmx'/>
    <feature policy='require' name='smx'/>
    <feature policy='require' name='est'/>
    <feature policy='require' name='tm2'/>
    <feature policy='require' name='xtpr'/>
    <feature policy='require' name='pdcm'/>
    <feature policy='require' name='pcid'/>
    <feature policy='require' name='dca'/>
    <feature policy='require' name='arat'/>
    <feature policy='require' name='pdpe1gb'/>
    <feature policy='require' name='rdtscp'/>
  </cpu>


Guest CPU Configuration that successfully migrates;

  <cpu mode='custom' match='exact'>
    <model fallback='allow'>Westmere</model>
    <topology sockets='1' cores='1' threads='1'/>
  </cpu>


Westmere Hosts CPU capabilities;

<cpu>
      <arch>x86_64</arch>
      <model>Westmere</model>
      <vendor>Intel</vendor>
      <topology sockets='1' cores='4' threads='2'/>
      <feature name='vme'/>
      <feature name='ds'/>
      <feature name='acpi'/>
      <feature name='ss'/>
      <feature name='ht'/>
      <feature name='tm'/>
      <feature name='pbe'/>
      <feature name='pclmuldq'/>
      <feature name='dtes64'/>
      <feature name='monitor'/>
      <feature name='ds_cpl'/>
      <feature name='vmx'/>
      <feature name='smx'/>
      <feature name='est'/>
      <feature name='tm2'/>
      <feature name='xtpr'/>
      <feature name='pdcm'/>
      <feature name='pcid'/>
      <feature name='dca'/>
      <feature name='arat'/>
      <feature name='pdpe1gb'/>
      <feature name='rdtscp'/>
      <feature name='invtsc'/>
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
      <pages unit='KiB' size=


Broadwell Host CPU Capabilities;

    <cpu>
      <arch>x86_64</arch>
      <model>Broadwell</model>
      <vendor>Intel</vendor>
      <topology sockets='1' cores='12' threads='2'/>
      <feature name='vme'/>
      <feature name='ds'/>
      <feature name='acpi'/>
      <feature name='ss'/>
      <feature name='ht'/>
      <feature name='tm'/>
      <feature name='pbe'/>
      <feature name='dtes64'/>
      <feature name='monitor'/>
      <feature name='ds_cpl'/>
      <feature name='vmx'/>
      <feature name='smx'/>
      <feature name='est'/>
      <feature name='tm2'/>
      <feature name='xtpr'/>
      <feature name='pdcm'/>
      <feature name='dca'/>
      <feature name='osxsave'/>
      <feature name='f16c'/>
      <feature name='rdrand'/>
      <feature name='arat'/>
      <feature name='tsc_adjust'/>
      <feature name='cmt'/>
      <feature name='xsaveopt'/>
      <feature name='mbm_total'/>
      <feature name='mbm_local'/>
      <feature name='pdpe1gb'/>
      <feature name='abm'/>
      <feature name='invtsc'/>
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
      <pages unit='KiB' size='1048576'/>
    </cpu>

Regards,
Oisin

Oisin O'Malley
Systems Engineer
Iocane Pty Ltd
763 South Road
Black Forest SA 5035

Office:+61 (8) 8413 1010
Fax:+61 (8) 8231 2050
Email:oisin.omalley@iocane.com.au
Web:www.iocane.com.au

Better for business

_______________________________________________
CentOS-virt mailing list
CentOS-virt@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos-virt

_______________________________________________
CentOS-virt mailing list
CentOS-virt@xxxxxxxxxx
https://lists.centos.org/mailman/listinfo/centos-virt

[Index of Archives]     [CentOS Users]     [Linux Media]     [Asterisk]     [DCCP]     [Netdev]     [X.org]     [Xfree86]     [Linux USB]

  Powered by Linux