Re: [PATCH v4 0/9] Selective block device migration implementation

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Tue, Jun 16, 2015 at 01:42:02AM +0300, Pavel Boldin wrote:

[. . .]

> Michal Privoznik (3):
>   virDomainDiskGetSource: Mark passed disk as 'const'
>   qemuMigrationBeginPhase: Fix function header indentation
>   qemuMigrationDriveMirror: Force raw format for NBD
> 
> Pavel Boldin (6):
>   util: multi-value virTypedParameter
>   util: multi-value parameters in virTypedParamsAdd*
>   util: virTypedParams{Filter,GetAllStrings}
>   util: add virTypedParamsAddStringList
>   qemu: migration: selective block device migration
>   virsh: selective block device migration
> 
>  include/libvirt/libvirt-domain.h |   9 ++
>  include/libvirt/libvirt-host.h   |  11 ++
>  src/conf/domain_conf.c           |   2 +-
>  src/conf/domain_conf.h           |   2 +-
>  src/libvirt_public.syms          |   6 +
>  src/qemu/qemu_driver.c           |  78 ++++++++---
>  src/qemu/qemu_migration.c        | 264 +++++++++++++++++++++++++----------
>  src/qemu/qemu_migration.h        |  24 ++--
>  src/util/virtypedparam.c         | 259 +++++++++++++++++++++++++++-------
>  src/util/virtypedparam.h         |  19 +++
>  tests/Makefile.am                |   6 +
>  tests/virtypedparamtest.c        | 295 +++++++++++++++++++++++++++++++++++++++
>  tools/virsh-domain.c             |  23 +++
>  tools/virsh.pod                  |  21 +--
>  14 files changed, 854 insertions(+), 165 deletions(-)
>  create mode 100644 tests/virtypedparamtest.c
> 

New test with this revision of patches applied.

Test env
~~~~~~~~

- On source and destination host, libvirt is compiled with the above
  patches:  

    $ git describe
    v1.2.16-204-g7aee251

- Create SSH keys and copy to dest host:

    # Create the SSH keys with empty passphrase
    $ ssh-keygen -t rsa
    # Copy the key to the remote host
    $ ssh-copy-id root@devstack3
    # `ssh root@devstack3` succeeds w/o password prompt

- Since I'm on a trusted network, on dest host:

    $ cat /etc/libvirt/libvirtd.conf | grep -v ^$ | grep -v ^#
    listen_tls = 0
    listen_tcp = 1
    auth_tcp = "none" 

- Run the libvirtd daemon on destination (with "--listen" mode), as
  root:

    $ ./run daemon/libvirtd --listen &


Test migration
~~~~~~~~~~~~~~

On source (from newly built libvirtd), as root:

    $ ./run tools/virsh list
 
I have two disks:

    $ ./run tools/virsh domblklist cvm1                                                            
    Target     Source
    ------------------------------------------------
    vda        /var/lib/libvirt/images/cirros-0.3.3-x86_64-disk.img
    vdb        /export/disk2.img

So, let's try to migrate the 'vdb' disk:

    $ ./virsh migrate --verbose --p2p  --migratedisks vdb \
        --live cvm1 qemu+ssh://root@devstack3/system
    error: Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainMigratePerform3Params)


>From libvirt debug logs
~~~~~~~~~~~~~~~~~~~~~~~

libvirtd debug log[1] from source (destination log is empty)):

[. . .]
2015-06-17 15:13:53.317+0000: 781: debug : virDomainMigratePerform3Params:5202 : dom=0x7f2118f13c40, (VM: name=cvm1, uuid=ab4c412b-6fdc-4fc4-b78c-f1d49db10d4e), dconnuri=qemu+tcp://root@devstack3/system, params=0x7f2118f12a90, nparams=1, cookiein=(nil), cookieinlen=0, cookieout=0x7f2106f38ba8, cookieoutlen=0x7f2106f38ba4, flags=3
2015-06-17 15:13:53.317+0000: 781: debug : virDomainMigratePerform3Params:5203 : params["migrate_disks"]=(string)vdb
2015-06-17 15:13:53.317+0000: 781: debug : qemuMigrationPerform:5238 : driver=0x7f20f416b840, conn=0x7f20dc005c30, vm=0x7f20f41e9640, xmlin=<null>, dconnuri=qemu+tcp://root@devstack3/system, uri=<null>, graphicsuri=<null>, listenAddress=<null>, nmigrate_disks=1, migrate_disks=0x7f2118f13930, cookiein=<null>, cookieinlen=0, cookieout=0x7f2106f38ba8, cookieoutlen=0x7f2106f38ba4, flags=3, dname=<null>, resource=0, v3proto=1
2015-06-17 15:13:53.317+0000: 781: debug : qemuDomainObjBeginJobInternal:1397 : Starting async job: none (async=migration out vm=0x7f20f41e9640 name=cvm1)
2015-06-17 15:13:53.317+0000: 781: debug : qemuDomainObjBeginJobInternal:1414 : Waiting for async job (vm=0x7f20f41e9640 name=cvm1)
2015-06-17 15:13:53.821+0000: 782: debug : virThreadJobSet:96 : Thread 782 (virNetServerHandleJob) is now running job remoteDispatchDomainGetJobInfo
2015-06-17 15:13:53.821+0000: 782: debug : virDomainGetJobInfo:8808 : dom=0x7f20dc008c30, (VM: name=cvm1, uuid=ab4c412b-6fdc-4fc4-b78c-f1d49db10d4e), info=0x7f2106737b50
2015-06-17 15:13:53.821+0000: 782: debug : virThreadJobClear:121 : Thread 782 (virNetServerHandleJob) finished job remoteDispatchDomainGetJobInfo with ret=0
2015-06-17 15:13:54.325+0000: 780: debug : virThreadJobSet:96 : Thread 780 (virNetServerHandleJob) is now running job remoteDispatchDomainGetJobInfo
2015-06-17 15:13:54.325+0000: 780: debug : virDomainGetJobInfo:8808 : dom=0x7f20dc008c30, (VM: name=cvm1, uuid=ab4c412b-6fdc-4fc4-b78c-f1d49db10d4e), info=0x7f2107739b50
2015-06-17 15:13:54.325+0000: 780: debug : virThreadJobClear:121 : Thread 780 (virNetServerHandleJob) finished job remoteDispatchDomainGetJobInfo with ret=0
[. . .]
remoteDispatchDomainMigratePerform3Params, 784 remoteDispatchDomainMigratePerform3Params) for (520s, 520s)
2015-06-17 15:14:23.320+0000: 781: error : qemuDomainObjBeginJobInternal:1492 : Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainMigratePerform3Params)
2015-06-17 15:14:23.320+0000: 781: debug : virThreadJobClear:121 : Thread 781 (virNetServerHandleJob) finished job remoteDispatchDomainMigratePerform3Params with ret=-1
2015-06-17 15:14:23.320+0000: 783: debug : virThreadJobSet:96 : Thread 783 (virNetServerHandleJob) is now running job remoteDispatchConnectClose
2015-06-17 15:14:23.320+0000: 783: debug : virThreadJobClear:121 : Thread 783 (virNetServerHandleJob) finished job remoteDispatchConnectClose with ret=0


How can I mitigate this? (I realize this is not due to these patches,
proably something with my test environment.)

Since this is non-shared storage migration, I tried to supply
'--copy-storage-inc' to no avail (same error as above).

Probably I should test by building local RPMs.

[1] https://kashyapc.fedorapeople.org/virt/temp/libvirtd-log-selective-blockdev-failed.log

-- 
/kashyap

--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list



[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]