On Tue, May 21, 2024 at 09:22:44 +0300, Dan Vatca via Users wrote: > Hello everybody, > > I am trying to understand how I can create a setup with two libvirt + kvm + > zfs systems, in which I can: > 1. Create a domain on sys-a, with storage space on a zfs filesystem. This > filesystem stores the qcow2 image, any qcow2 disk snapshots, memory dumps > and domain XML definition. > 2. Take consistent ZFS snapshots every 15 minutes, and sync them to sys-b. > The snapshots will be replicated using syncoid. These will be using scripts > to quiesce the VM. > 3. Given that most of the disk is synced (at most 15 minutes ago), what are > the things I need to ensure (libvirt snapshots, memory snapshot, etc), so > that I can do a live migration while copying just the incremental changes > since the last snapshot I already transferred? > > I was trying to use virsh migrate —copy-storage-inc, however I could not > find any working example that would copy just the incremental changes. I > could find the theory of the snapshot backing chain, however Was this theory involving creation of an external snapshot (new qcow2 overlay), then ensuring that the original image is replicated to the other host and then using migration with incremental copy? If yes, the theory is correct, but requires creating the snapshot, without actual snapshot metadata (just creating the qcow2 overlay). This is done via 'virsh snapshot-create --no-metadata'. This does everything to do the snapshot, but doesn't actually record it in the list of snapshots ... Alternatively you can delete the sanpshot metadata if you have them already around (virsh snapshot-delete --metadata). > —copy-storage-inc complains that it cannot replicate while there are > snapshots present. (error: Requested operation is not valid: cannot migrate > domain with 1 snapshots). ... bypassing this check. Now this check has some history behind it, the gist being that libvirt doesn't actually copy the metadata over, so any snapshots are unusable on the destination. Additionally when using --copy-storage-all the disk image chain is flattened, thus snapshots are not preserved. Reallistically libvirt should reconsider this check, which is more than 10 years old, as it in certain cases it'd make sense to migrate the VM even without preserving snapshot state. Or in cases of shared storage replicating the snapshot metadata would certainly improve usability.
