> On 10 Jan 2025, at 12:23, Daniel P. Berrangé <berrange@xxxxxxxxxx> wrote: > > On Fri, Jan 10, 2025 at 12:09:44PM +0000, Felipe Franciosi wrote: >> >> >>> On 10 Jan 2025, at 11:00, Daniel P. Berrangé <berrange@xxxxxxxxxx> wrote: >>> >>> On Fri, Jan 10, 2025 at 10:49:20AM +0000, Felipe Franciosi wrote: >>>> >>>> >>>>> On 10 Jan 2025, at 08:33, Daniel P. Berrangé <berrange@xxxxxxxxxx> wrote: >>>>> >>>>> On Thu, Jan 09, 2025 at 07:27:16PM +0000, Felipe Franciosi wrote: >>>>>> Hello! >>>>>> >>>>>> I have a use case which I'm struggling to support with libvirt: >>>>>> saving a VM to a file, cloning it (which renames the VM), and restoring it. >>>>>> >>>>>> My search revealed a number of tutorials for using virt-clone [1], but that >>>>>> doesn't seem to cover VMs which are _saved_ (only running or paused). >>>>> >>>>> Saved in what way ? Managed save ? >>>> >>>> Thanks for the prompt reply! >>>> >>>> I'm saving with virDomainSave(). My understanding is that this is not managed. >>> >>> Functionally it is the same as managed save, just the that file path >>> is specified by the client, rather than by libvirt. >> >> Got it, thanks. >> >>>>>> [1] https://github.com/virt-manager/virt-manager/blob/main/virtinst/virtclone.py >>>>>> >>>>>> In a nutshell, I want to power on a VM and do some setup, then save its full >>>>>> state to disk (e.g., with virsh save). Finally I want to modify the XML to: >>>>>> - rename the VM >>>>>> - change which bridge its NICs are on (while maintaining mac addresses) >>>>>> - change the disk image to a copy (done while the VM is saved) >>>>>> >>>>>> But the restore operation fails because of a target domain name check >>>>>> implemented in virDomainDefCheckABIStabilityFlags(). I've debated how to best >>>>>> address this and I'm looking for your views. >>>>> >>>>> If you're cloning a VM, it needs both a new UUID and name, so I'm surprised >>>>> the ABI stability check hasn't already blocked you on the UUID change before >>>>> getting to the name change check. >>>> >>>> I definitely didn't change the UUID. In fact, I want it to be the same (at >>>> least in the SMBIOS tables) because the guest OS is not going to expect that >>>> value to change without a power cycle/reset. The ABI Check actually ensures the >>>> SMBIOS values do not change during restore. >>>> >>>> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0b53da1/src/conf/domain_conf.c#L21759 >>>> >>>> My understanding is that this passed because the other domain was not running >>>> (and the save was unmanaged, so libvirt is unaware of the saved VM). >>>> >>>> What I don't understand is why the UUID has to be unique (or, in fact, the same >>>> as the SMBIOS Type 1 UUID). Isn't this something just visible to the VM? For >>>> the clone use case, I surely don't want this to change. >>>> >>>> In other words, it's not clear to me why this check is needed: >>>> https://gitlab.com/libvirt/libvirt/-/blob/caa10431cdd1aa476637ff721f1947c4e0b53da1/src/conf/domain_conf.c#L12810 >>> >>> Libvirt has three unique identifiers for all VMs - UUID, name, and ID. The >>> latter is only for running VMs. >>> >>> UUID is the primary unique identifier that is used for pretty much every >>> lookup inside libvirt. Name is a secondary unique identifier largely just >>> for external lookups by humans, since UUIDs are not human friendly. >>> >>> Essentially every API call starts with virDomainObjListFindByUUID to convert >>> the public 'virDomainPtr' object into the internal 'virDomainObjPtr' struct >>> that holds the config & state. >> >> Ah-ha. Ok, this is really helpful, thanks again! >> >> My next question is why the SMBIOS Type 1 UUID tied to the Libvirt identifier? >> (I'm pointing again at L#12810 above.) >> >> That feels incorrect. My (new) understanding is that: >> - The SMBIOS Type 1 UUID is guest-visible >> - The Libvirt UUID is a host identifier >> >> What comes to mind is that maybe something like guest tools wants to be able to >> report back to a control plane what VM it is on based on this value. If that's >> the motivation, then isn't Generation ID a better field to rely on? > > Strictly speaking we don't have to tie them together, but in practice we > do, because it is pretty compelling to be correlate data between the host > OS and guest OS for apps. Right, so libvirt uses the XML <UUID> as a host unique identifier and also as QEMU's "-uuid" parameter (which, based on my understanding, is a default for any virtual hardware UUID stuff such as SMBIOS Type 1 UUID/Serial. And the reason for that is to allow guests to infer their hypervisor identifier. > >> My understanding is that SMBIOS identifiers cannot change at runtime. > > Correct. Ok, but then I'm confused: how is clone supposed to work? When I clone a saved VM, Libvirt requires that I change its <UUID> and it also requires that this <UUID> matches SMBIOS Type 1 UUID which, by definition, cannot change. What am I missing? >>> The ABI stability checks are done in places where we allow the XML of an >>> existing VM to be replaced, and that doesn't involve calls to virDomainObjListAdd >>> since we're not creating a new VM, just altering its backend config. >> >> So maybe the problem I have is that I'm not modifying this data when I clone >> the saved VM? To recap, I'm calling virDomainSave() and giving it a path. The >> path creates a file named after the VM. I'm just renaming that file. Maybe I >> should be modifying the XML embedded in that file somehow? > >> >>> >>>>> Likewise for UUID checks. >>>> >>>> I still don't understand how UUID is used, so clarification/pointers welcome! >>>> >>>>>> I'm open to suggestions, for example by plumbing through a flag which makes the >>>>>> check optional. Please let me know how you prefer that I take this forward. >>>>> >>>>> If you're using managed save, then I would think it is already possible to >>>>> achieve. >>>>> >>>>> First cloning the VM: >>>>> >>>>> virsh dumpxml myvm > myvm.xml >>>>> cp myvm.xml myvm-clone.xml >>>>> ..modify name & uuid & bridge & disk of myvm-clone.xml >>>>> virsh define myvm-clone.xml >>>>> >>>>> Now modify the saved state >>>>> >>>>> virsh managedsave-dumpxml myvm-clone > save.xml >>>>> ...modify save.xml to change name & uuid to match myvm-clonme... >>>>> virsh managedsave-define save.xml >>>>> >>>>> The ABI stability check done by managedsave-define, will validate against >>>>> the name + uuid of the cloned VM, so in fact it will force you to change >>>>> the name + uuid in the save XML before loading it back in, and it should >>>>> be not possible to restore from the managed save image until fixing this >>>> >>>> Thanks for these pointers; I'll look into them. But I think my workflows are >>>> not managed as these VMs are not persistent on the host. When they die, they >>>> may be restarted elsewhere by a higher-level (multi-host) control plane. >>> >>> There are equivalent save-image-define / save-image-dumpxml for the >>> non-managed save scenario. >> >> And presumably this is how I get a saved XML which I can then modify? > > Yes, that's the supported way of modifying the XML inside a save image, > other than editting the file directly (please don't :-). lol what could possibly go wrong with opening a binary file, inferring some offsets, and patching it? j/k :) F.
