On 04.11.2022 20:19, Andrea Bolognani wrote:
On Fri, Nov 04, 2022 at 01:03:52PM +0100, Peter Krempa wrote:
On Fri, Nov 04, 2022 at 16:43:00 +0600, Oleg Vasilev wrote:
Hotplugging PCI devices on pc-i440fx machines is supported without
additional configuration. On q35, pcie-to-pci-bridge needs to be added
prior to the machine startup in order to support hotplug [1].
The idea is to support the same workflow for creating VMs in q35, as was
in pc. Namely, do not require additional configuration when hotplugging
is needed. Otherwise, all libvirt clients need to be updated which there
are a lot and they are maintained by different parties.
Instead, a pcie-to-pci-bridge better be created by default, so that PCI
slots are readily available. Might be a good idea to make it configurable
in the future.
Previously there was a pci-bridge present by default, but was removed in
d5fb8f4564 (qemu: don't add pci-bridge to Q35/arm domains unless it's needed, 2016-04-22)
[1]: https://libvirt.org/pci-hotplug.html
+ /* Add a pcie-to-pci-bridge and pcie-root-port to plug it into. */
+ if (addPCIeToPCIBridge) {
+ if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 1,
+ VIR_DOMAIN_CONTROLLER_MODEL_PCIE_ROOT_PORT) < 0)
+ return -1;
+ if (virDomainDefMaybeAddController(def, VIR_DOMAIN_CONTROLLER_TYPE_PCI, 2,
+ VIR_DOMAIN_CONTROLLER_MODEL_PCIE_TO_PCI_BRIDGE) < 0)
+ return -1;
+ }
Doing this unconditionally breaks ABI of the VM, as new guest visible
devices appear without any change in configuration. What's worse this
implementation would also break migration compatibility from older
versions of libvirt as there isn't any form of explicit handling for it.
I unfortunately don't see any reasonable solution originating in
libvirt. The bridge will need to be added by users or higher level
management tools (e.g. virt-install) as libvirt explicitly wants to
support use cases when VMs are started without persisting the XML
definition (e.g. always re-define the XML before start or use the
'createXML' api).
Additionally, after this change it would be impossible to create a
q35 VM *without* the additional bridge. Most users of the q35 machine
type are likely using PCIe devices exclusively, and such a change
would negatively impact them.
In the (hopefully rare) cases in which hotplugging traditional PCI
devices is actually required, enabling that at the time when the
domain is defined is already straightforward: just include the
relevant controller in the XML.
Unfortunately q35, unlike i440fx, doesn't support hotplugging of any
kind out of the box, so it's up to the user / management application
to ensure that the necessary controllers are added. This is not
ideal, but there's no way around it, and it's still preferable to
forcing unavoidable, potentially useless extra controllers on
everybody.
Apparently, there is a way to hotplug pcie-to-pci-bridge described in
qemu doc [1]. Do you think this approach with bus-reserve=1 would be
acceptable to support in libvirt by default?
[1]: https://github.com/qemu/qemu/blob/master/docs/pcie_pci_bridge.txt#L25
