Hello,
I am trying to understand libvirt's logic for checking whether migration
of a VM is safe, and how it determines which disks should be mirrored by
QEMU. My particular use case involves VMs that may have disks backed onto
LVM or onto Ceph RBD, or both.
As far as I can tell, the qemuMigrationIsSafe check is there to ensure
that all disks are readonly, or have cache=none, or their backends can
guarantee cache coherence. As far as I can tell, however, QEMU flushes
*all* block devices when it pauses a VM's CPUs (just before the final part
of migration, for instance), so I'm wondering why this check is needed. Is
there any possible situation for the source VM to be paused, for its block
devices to be flushed, and yet the destination VM can't see all completed
writes?
Why is RBD is handled specially in this function? The current logic is
that an RBD-backed disk is safe to be migrated even if it's got caching
enabled, but I'm not sure how RBD is different from other backends in this
regard.
If VIR_MIGRATE_NON_SHARED_DISK or _INC is specified, should these safety
checks be relaxed? It seems to me that if any non-shared disk is going to
be *explicitly* copied from the source to the destination VM, then cache
coherence in the backend is irrelevant.
At the moment, the set of non-shared block devices copied by
VIR_MIGRATE_NON_SHARED_* differs depending on whether NBD is being used in
the migration:
- If NBD can't be used (e.g. with a tunnelled migration), then QEMU will
copy *all* non-readonly block devices;
- If NBD is being used, then QEMU will only mirror "shareable", "readonly"
or "sourceless" disks.
A problem arises with RBD disks that have caching enabled. According to
qemuMigrationIsSafe, these disks are "safe" to be migrated. However in
either the NBD or the non-NBD case, the RBD disk will be copied. This is
clearly not desirable. If RBD is a special case in qemuMigrationIsSafe,
does it also need to be a special case when configuring the NBD server?
Or, if an NBD server is not going to be used, should the migration be
considered "unsafe" if an RBD disk is present?
I'd very much appreciate some help in understanding all of this. At the
moment, I think my only option is to run RBD without caching at all.
However, not only does that result in very poor performance, it also
doesn't seem to match the qemuMigrationIsSafe check.
Regards,
Michael
--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list