On 2014-09-24 14:32, Jiri Denemark wrote:
On Tue, Sep 23, 2014 at 16:09:57 +0200, Cristian Klein wrote:Added a new `libvirt` migration flag `VIR_MIGRATE_POSTCOPY` and the necessary code to pass this flag to qemu as migration capability. Signed-off-by: Cristian Klein <cristian.klein@xxxxxxxxx> --- include/libvirt/libvirt.h.in | 1 + src/libvirt.c | 7 +++++++ src/qemu/qemu_migration.c | 43 +++++++++++++++++++++++++++++++++++++++++++ src/qemu/qemu_migration.h | 3 ++- 4 files changed, 53 insertions(+), 1 deletion(-) diff --git a/include/libvirt/libvirt.h.in b/include/libvirt/libvirt.h.in index 6371b7b..bdc33c6 100644 --- a/include/libvirt/libvirt.h.in +++ b/include/libvirt/libvirt.h.in @@ -1225,6 +1225,7 @@ typedef enum { VIR_MIGRATE_ABORT_ON_ERROR = (1 << 12), /* abort migration on I/O errors happened during migration */ VIR_MIGRATE_AUTO_CONVERGE = (1 << 13), /* force convergence */ VIR_MIGRATE_RDMA_PIN_ALL = (1 << 14), /* RDMA memory pinning */ + VIR_MIGRATE_POSTCOPY = (1 << 15), /* enable (but don't start) post-copy */ } virDomainMigrateFlags; diff --git a/src/libvirt.c b/src/libvirt.c index 1a285ca..33aeafa 100644 --- a/src/libvirt.c +++ b/src/libvirt.c @@ -5265,6 +5265,7 @@ virDomainMigrateDirect(virDomainPtr domain, * automatically when supported). * VIR_MIGRATE_UNSAFE Force migration even if it is considered unsafe. * VIR_MIGRATE_OFFLINE Migrate offline + * VIR_MIGRATE_POSTCOPY Enable (but don't start) post-copy * * VIR_MIGRATE_TUNNELLED requires that VIR_MIGRATE_PEER2PEER be set. * Applications using the VIR_MIGRATE_PEER2PEER flag will probably @@ -5291,6 +5292,12 @@ virDomainMigrateDirect(virDomainPtr domain, * can use either VIR_MIGRATE_NON_SHARED_DISK or * VIR_MIGRATE_NON_SHARED_INC as they are mutually exclusive. * + * If you want to enable post-copy migration you must set the + * VIR_MIGRATE_POSTCOPY flag. Once migration is active, you may + * start post-copy by calling virDomainMigrateStartPostCopy. + * When to start post-copy is entirely left to the user, libvirt + * only implements the necessary mechanism. + * * In either case it is typically only necessary to specify a * URI if the destination host has multiple interfaces and a * specific interface is required to transmit migration data. diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c index a5bd825..bd1f2d6 100644 --- a/src/qemu/qemu_migration.c +++ b/src/qemu/qemu_migration.c @@ -1799,6 +1799,45 @@ qemuMigrationSetOffline(virQEMUDriverPtr driver, static int +qemuMigrationSetPostCopy(virQEMUDriverPtr driver, + virDomainObjPtr vm, + qemuDomainAsyncJob job)s/ // in the two lines above to fix indentation.
Will fix in next version of the patch. Thanks for pointing out "make syntax-check".
+{ + qemuDomainObjPrivatePtr priv = vm->privateData; + int ret; + + if (qemuDomainObjEnterMonitorAsync(driver, vm, job) < 0) + return -1; + + ret = qemuMonitorGetMigrationCapability( + priv->mon, + QEMU_MONITOR_MIGRATION_CAPS_POSTCOPY);QEMU_MONITOR_MIGRATION_CAPS_POSTCOPY is not defined anywhere in this patchset. It seems you forgot to update qemu_monitor.[ch].
Sorry about that, I divided the patches incorrectly. Will be fixed in PATCH v2.
+ + if (ret < 0) { + goto cleanup; + } else if (ret == 0) { + if (job == QEMU_ASYNC_JOB_MIGRATION_IN) { + virReportError(VIR_ERR_ARGUMENT_UNSUPPORTED, "%s", + _("Post-copy migration is not supported by " + "target QEMU binary")); + } else { + virReportError(VIR_ERR_ARGUMENT_UNSUPPORTED, "%s", + _("Post-copy migration is not supported by " + "source QEMU binary")); + } + ret = -1; + goto cleanup; + } + + ret = qemuMonitorSetMigrationCapability( + priv->mon, + QEMU_MONITOR_MIGRATION_CAPS_POSTCOPY); + + cleanup: + qemuDomainObjExitMonitor(driver, vm); + return ret; +} +static int qemuMigrationSetCompression(virQEMUDriverPtr driver, virDomainObjPtr vm, qemuDomainAsyncJob job) @@ -3580,6 +3619,10 @@ qemuMigrationRun(virQEMUDriverPtr driver, if (flags & VIR_MIGRATE_RDMA_PIN_ALL && qemuMigrationSetPinAll(driver, vm, QEMU_ASYNC_JOB_MIGRATION_OUT) < 0) + + if (flags & VIR_MIGRATE_POSTCOPY && + qemuMigrationSetPostCopy(driver, vm, + QEMU_ASYNC_JOB_MIGRATION_OUT) < 0) goto cleanup; if (qemuDomainObjEnterMonitorAsync(driver, vm,I'd expect similar thing would need to be done in the Prepare phase on destination... However, if destination does not need to set the capability, we at least need to check if destination QEMU supports it and report failure from Prepare if it doesn't. And the QEMU_ASYNC_JOB_MIGRATION_IN branch in qemuMigrationSetPostCopy would be unreachable in that case.
It's up to the source qemu (through the migration protocol) to activate post-copy on the destination qemu. So there are no other steps to be done on the source.
I have mixed feelings about having libvirt check the necessary capability on the destination. Personally, I would prefer qemu to return an error like "post-copy unavailable" when the "migrate" command is issued, as there might be more factors that influence the availability of post-copy. For example, it's not sufficient for the destination qemu to support post-copy, but the destination kernel also needs userfault support.
Dave, I just tried migration from the post-copy qemu to master qemu and it fails unexpectedly. The "migrate" command return no error, but a "failed" migration status is returned later. Weirdly, this also happens when I initiate non-post-copy migration. Do you happen to know what is the qemu policy regarding migration? What would be the best way to determine if a certain type of migration is feasible, while at the same time returning a meaningful error to the user?
diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h index e7a90c3..349c9c4 100644 --- a/src/qemu/qemu_migration.h +++ b/src/qemu/qemu_migration.h @@ -41,7 +41,8 @@ VIR_MIGRATE_COMPRESSED | \ VIR_MIGRATE_ABORT_ON_ERROR | \ VIR_MIGRATE_AUTO_CONVERGE | \ - VIR_MIGRATE_RDMA_PIN_ALL) + VIR_MIGRATE_RDMA_PIN_ALL | \ + VIR_MIGRATE_POSTCOPY) /* All supported migration parameters and their types. */ # define QEMU_MIGRATION_PARAMETERS \
-- Cristian Klein, PhD Post-doc @ Umeå Universitet http://www8.cs.umu.se/~cklein -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list