Daniel,
I looked at the patch-set you sent out on the 2/9/11
[PATCH 0/6] Introduce a new migration protocol
to QEMU driver
http://www.mail-archive.com/libvir-list@xxxxxxxxxx/msg33223.html
What is the status of this new migration protocol?
Is there any pending issue blocking its integration?
I would like to propose an RFC enhancement to the migration
algorithm.
Here is a quick summary of the proposal/idea.
- finer control on migration result
- possibility of specifying what features cannot fail
their initialization on the dst host during migration.
Migration should not succeed if any of them fails.
- optional: each one of those features should be able to
provide a deinit function to cleanup resources
on the dst host if migration fails.
This functionality would come useful for the (NIC) set port
profile feature VDP (802.1Qbg/1Qbh), but what I propose is
a generic config option / API that can be used by any feature.
And now the details.
----------------------------------------------
enhancement: finer control on migration result
----------------------------------------------
There are different reasons why a VM may need (or be forced) to
migrate.
You can classify the types of the migrations also based on
different semantics.
For simplicity I'll classify them into two categories, based on
how important it is for the VM to migrate as fast as possible:
(1) It IS important
In this case, whether the VM will not be able to (temporary)
make use of certain resources (for example the network) on the
dst host, is not that important, because the completion of the
migration is considered higher priority.
A possible scenario could be a server that must migrate ASAP
because of a disaster/emergency.
(2) It IS NOT important
I can think of a VM whose applications/servers need a network
connection in order to work properly. Loosing such network
connectivity as a consequence of a migration would not be
acceptable (or highly undesirable).
Given the case (2) above, I have a comment about the Finish
step, with regards to the port profile (VDP) codepath.
The call to
qemuMigrationVPAssociatePortProfile
in
qemuMigrationFinish
can fail, but its result (success or failure) does not influence
the result of the migration Finish step (it was already like this
in migration V2).
It is therefore possible for a VM to lose its network connectivity
after a (successful) migration.
Given that you are in the process of adding a new migration
version, I was wondering if we could add a mechanism for the src
host to select behavior/policy (1) or (2).
It would be one more option in the libvirt XML domain config file.
The above example refers to one specific feature (network connectivity
based on the use of a port profile, ie VDP), but in theory this
feature could be made generic such that the same policy (1)/(2)
could be used for any feature that the admin may not be willing
to lose as result of a (successful) migration.
The management application on the src host (virt-manager, RHEV-M, etc)
can for example, in case (2), retry the migration toward the same
host or try with another destination host.
The port profile configuration happens in two stages:
- there is a 1st call during PREPARE:
openMacvtaptap
|
+-> (a) vpAssociatePortProfileId
(PREASSOCIATE)
- and a second call during FINISH:
qemuMigrationVPAssociatePortProfile
|
+-> (b) vpAssociatePortProfileId
(ASSOCIATE)
Right now, both in migration V2 and V3:
- (a) cannot fail (ie, if it fails, migration fails too),
- (b) can fail (ie, if it fails, migration does not
fail and VM loses connectivity).
With the introduction of the policy (1)/(2) mentioned
above, this would be the new behavior:
- Case (1) above, ie fast migration
--> (a) and (b) are non blocking
--> Their result does not affect migration result.
- Case (2) above, ie possibly slower migration
--> (a) and (b) are blocking
--> Their result does affect migration result.
It may be not possible to handle all possible scenarios (there
can be cases where dependencies between features would make it
difficult to chose between (1) and (2) on a per feature
basis), but that should not be a reason not to provide this
policy for a subset of features (network connectivity being
one).
The 5-steps new migration version V3 makes it easier
(compared to V2) to introduce this kind of policy.
I guess we could use the cookies to carry the necessary
information/acks.
There are still numerous things that can go wrong.
For example, if migration aborts after (a), or even
after (b), there should be a way for Libvirt to undo the
set port profile.
Actually, there should be a way for any feature (not just
VDP) that successfully initialized on the dst host to
(at least try to) cleanup.
For this reason, it could make sense, for each one of the
features using policy (2) (ie, cannot fail during migration)
to also provide a deinit routine (as part of the API that
marks the feature as "can't fail").
Such routine/s can be called by libvirt to revert/rollback
the initializations that successfully completed during the
migration attempt.
This could be done probably at the end of the Finish3 phase.
If something goes wrong (including a signal that aborts
migration just before Confirm3 completes), we may have a
small problem. I do not know how we can handle that case
without another cleanup RPC call.
If you agree that this kind of functionality/policy described
above has some value, I can help with its implementation and
testing.
BTW, would the new functionality being discussed in this 3D
"RFC: virInterface change transaction API"
http://www.redhat.com/archives/libvir-list/2011-April/msg00499.html
be able to provide the same configuration "atomicity" (ie, rollback
in case of migration failure)?
My understanding is that:
- Such new framework would apply to (host) network config only.
Even though it may cover the VDP (port profile) use case I
mentioned above, it would not apply to other features that
may need some kind of rollback after a migration failure.
- Would it be possible to design that new transaction API
(or a sister API) such that it would come useful for the
guest migration case too? If so, the deinit/rollback routine
I proposed may be defined as a simple wrapper around the
transaction API and let other (non networking) features
use their own deinit functions.
Thanks
/Chris
> -----Original Message-----
> From: libvir-list-bounces@xxxxxxxxxx [mailto:libvir-list-
> bounces@xxxxxxxxxx] On Behalf Of Eric Blake
> Sent: Tuesday, April 12, 2011 4:12 PM
> To: Daniel P. Berrange
> Cc: libvir-list@xxxxxxxxxx
> Subject: Re: [PATCH 3/6] Introduce yet another migration
> version in API.
>
> On 02/09/2011 09:58 AM, Daniel P. Berrange wrote:
> > This patch attempts to introduce a version 3 that uses the improved
5
> > step sequence
> >
> > * Src: Begin
> > - Generate XML to pass to dst
> > - Generate optional cookie to pass to dst
> >
> > * Dst: Prepare
> > - Get ready to accept incoming VM
> > - Generate optional cookie to pass to src
> >
> > * Src: Perform
> > - Start migration and wait for send completion
> > - Generate optional cookie to pass to dst
> >
> > * Dst: Finish
> > - Wait for recv completion and check status
> > - Kill off VM if failed, resume if success
> > - Generate optional cookie to pass to src
> >
> > * Src: Confirm
> > - Kill off VM if success, resume if failed
>
> I've been thinking about this a bit more, and have a question. What
> happens when the source side decides to abort the migration? For
> example, if libvirtd on the source gets a SIGHUP or SIGINT, it would
be
> nice to have the cleanup code abort any in-flight migrations so that
> when libvirtd is restarted, the guest is still operational on the
> source, and the guest does not have to wait for a full TCP timeout
> cycle to realize that the source is not going to complete the
> migration.
>
> Does this call for additional internal points in the RPC
implementation
> of v3 migration?
>
> --
> Eric Blake eblake@xxxxxxxxxx +1-801-349-2682
> Libvirt virtualization library http://libvirt.org
--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list