Re: Predictable and consistent net interface naming in guests

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]


On 10/31/22 2:21 PM, Edward Haas wrote:

On Mon, Oct 31, 2022 at 6:55 PM Andrea Bolognani <abologna@xxxxxxxxxx <mailto:abologna@xxxxxxxxxx>> wrote:

    On Mon, Oct 31, 2022 at 04:32:27PM +0200, Edward Haas wrote:
     > That discussion mentioned that a guest PCI address may change in
    two cases:
     > - The PCI topology changes.
     > - The machine type changes.
     > Usually, the machine type is not expected to change, especially
    if one
     > wants to allow migrations between nodes.
     > I would hope to argue this should not be problematic in practice,
     > guest images would be made per a specific machine type.

    The machine type might not change from q35 to i440fx and vice versa,
    but since the domain XML is constructed every time a KubeVirt VM is
    started, the machine type might be q35-6.0 on one boot and q35-7.0
    the next one if a KubeVirt upgrade that comes with a new version of
    QEMU has happened in between.

    This is unlikely to make a difference in terms of PCI addresses seen
    in the guest OS, but it's still not accurate to say that the machine
    type will not change.

Thank you for the clarification.
It makes me wonder now what are the actual implications of
the machine type change.

    Live migration is a separate matter, as the machine type will
    definitely not change while the VM is running.

     > Regarding the PCI topology, I am not sure I understand what changes
     > need to occur to the domxml for a defined guest PCI address to
     > The only think that I can think of is a scenario where
    hotplug/unplug is
     > used,
     > but even then I would expect existing devices to preserve their
    PCI address
     > and the plug/unplug device to have a reserved address managed by
    the one
     > acting on it (the management system).
     > Could you please help clarify in which scenarios the PCI topology
    can cause
     > a mess to the naming of interfaces in the guest?

    A change in libvirt (again, due to a KubeVirt upgrade in between two
    boots of the same VM) might result in different PCI addresses being
    assigned to devices despite the same input XML.

    We generally try fairly hard to avoid this kind of situation, but we
    can only really guarantee stable PCI addresses for the lifetime of a
    VM that has been defined and can't promise that the same input XML
    will result in the same guest ABI when using different versions of

I would expect the PCI addresses that have been explicitly set in the
domxml [2] to be honored. We cannot assume that?

*If* the PCI address has been set in the original XML, that address will be honored any and every time a new domain is defined from that XML. Alternately, if a domain is defined once (without explicitly specifying any PCI addresses) and then run multiple times from the same definition, libvirt will auto-generate PCI addresses at initial definition time, and then use those same addresses each time the domain is run.

The issue is that no management application, including KubeVirt, is explicitly setting the PCI addresses of devices (and we believe that hands-off practice should continue), *AND* KubeVirt is re-defining the domain each time it is run (without querying libvirt for (and so never saving) the PCI addresses that were assigned to the devices. So each time the domain is stopped, all the PCI address info from that run is thrown away. And each time the domain is re-started (by re-defining it from the original XML that has no PCI address info), libvirt starts from scratch assigning addresses based on the information it receives from KubeVirt. And if the conditions have changed, then addresses are assigned differently.

The potential situation Andrea described, where the PCI addresses could be changed merely due to an upgrade of KubeVirt/libvirt/qemu from one run to the next in spite of being fed the same (adress-less) XML, is actually extremely rare (I don't remember such a case) but theoretically it could happen. The more common change would be if a device was added or removed during one run of the guest, and then remained added/removed the next time it was run - that could change the PCI addresses of one or more of the remaining devices, depending on their ordering in the XML).

So, libvirt provides two avenues to maintaining stable PCI addresses (and thus, network device names) across multiple runs of a domain (either define once, run many, or else query the XML of the running domain and use that XML (containing PCI addresses) the next time the domain is started, but KubeVirt doesn't use either of these (and if memory serves me correctly, it really can't due to its design. And delegating management of PCI addresses to KubeVirt is pushing too much complexity out to KubeVirt.

I mainly referred to that input option, not to the expectation that the generated
configuration (of the domxml) to be identical between different versions.

[2] <>

-- Andrea Bolognani / Red Hat / Virtualization

[Index of Archives]     [Virt Tools]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Yosemite News]     [KDE Users]

  Powered by Linux