When faced with a guest device that requires a PCI address but doesn't have one manually assigned in the config, libvirt has always insisted (well... *tried* to insist) on auto-assigning an address that is on a PCI controller that supports hotplug. One big problem with this is that it prevents automatic use of a Q35 (or aarch64/virt) machine's pcie-root (since the PCIe root complex doesn't support hotplug). In order to promote simpler domain configs (more devices on pcie-root rather than on a pci-bridge), this patch adds a new sub-element to all guest devices that have a PCI address and support hotplug: <hotplug require='no'/> For devices that have hotplug require='no', we turn off the VIR_PCI_CONNECT_HOGPLUGGABLE bit in the devFlags when searching for an available PCI address. Since pcie-root now allows standard PCI devices, this results in those devices being placed on pcie-root rather than pci-bridge. NB: According to the code in qemuDomainAssignDevicePCISlots() and qemuDomainAttachDeviceLive(), the following device types are the only ones that use a PCI address *and* support hotplug: <interface> <hostdev> <rng> <serial> <disk> (only virtio) <controller> (hotplug only for SCSI controllers) These devices use a PCI address, but don't support hotplug, so the RNG doesn't allow them to have a <hotplug> element: <filesystem> <sound> <balloon> <watchdog> <video> <shmem> <input> --- docs/formatdomain.html.in | 37 ++++++++ docs/schemas/domaincommon.rng | 17 ++++ src/conf/device_conf.h | 15 +++ src/conf/domain_addr.c | 7 ++ src/conf/domain_conf.c | 28 ++++++ .../qemuxml2argv-q35-hotpluggable.args | 50 ++++++++++ .../qemuxml2argv-q35-hotpluggable.xml | 62 +++++++++++++ tests/qemuxml2argvtest.c | 8 ++ .../qemuxml2xmlout-q35-hotpluggable.xml | 103 +++++++++++++++++++++ tests/qemuxml2xmltest.c | 5 + 10 files changed, 332 insertions(+) create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-hotpluggable.args create mode 100644 tests/qemuxml2argvdata/qemuxml2argv-q35-hotpluggable.xml create mode 100644 tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-hotpluggable.xml diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in index eddb8dd..d5292f1 100644 --- a/docs/formatdomain.html.in +++ b/docs/formatdomain.html.in @@ -3038,6 +3038,43 @@ assign a PCI address for the device rather than some other type of address that may also be appropriate for that same device (e.g. virtio-mmio). + <p> + If a device that needs a PCI address doesn't have one + explicitly provided in the XML configuration, libvirt will + automatically assign an appropriate address before saving + the configuration. During this automatic assignment, if a + device supports being hotplugged/unplugged (attaching and + detaching the device while the virtual machine is running) + libirt's default behavior is to assign that device to a slot + on a PCI controller that also supports having its devices + hotplugged/unplugged. <i>(In the qemu driver as of libvirt + 2.2.0, these devices can be hotplugged: <interface>, + <hostdev>, <rng>, <disk> (virtio-only), + <controller> (SCSI controllers only), and + <serial> (PCI serial only). Likewise, the following + PCI controller models support having their connected devices + hotplugged: pci-root, pci-bridge, pci-expander-bus, + pcie-root-bus, and pcie-switch-downstream-port.)</i> To allow + auto-assigning hotpluggable devices to PCI controllers that + don't support hotplug (for example, to assign the device to + pcie-root), those devices that support hotplug can have an + optional + <code><hotplug></code> element with a single + attribute <code>require</code> that can be used to inform + libvirt that this particular device <b>will not</b> require + hotplugging; libvirt will then look for open slots in all + appropriate controllers regardless of whether or not they + support hotplug. + </p> + <pre> + ... + <interface type='network'> + <source network='default'/> + <mac address='52:54:00:5d:c7:9e'/> + <hotplug require='no'/> + </interface> + ...</pre> + </dd> <dt><code>drive</code></dt> <dd>Drive addresses have the following additional diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng index 052f28c..e45bec1 100644 --- a/docs/schemas/domaincommon.rng +++ b/docs/schemas/domaincommon.rng @@ -1171,6 +1171,7 @@ <optional> <ref name="alias"/> </optional> + <ref name="hotplug"/> <optional> <ref name="address"/> </optional> @@ -1745,6 +1746,7 @@ <optional> <ref name="alias"/> </optional> + <ref name="hotplug"/> <optional> <ref name="address"/> </optional> @@ -2619,6 +2621,7 @@ <optional> <ref name="alias"/> </optional> + <ref name="hotplug"/> <optional> <ref name="address"/> </optional> @@ -3274,6 +3277,7 @@ <optional> <ref name="alias"/> </optional> + <ref name="hotplug"/> <optional> <ref name="address"/> </optional> @@ -3877,6 +3881,7 @@ <optional> <ref name="rom"/> </optional> + <ref name="hotplug"/> <optional> <ref name="address"/> </optional> @@ -4717,6 +4722,17 @@ </element> </define> + <define name="hotplug"> + <optional> + <element name="hotplug"> + <attribute name="require"> + <ref name="virYesNo"/> + </attribute> + <empty/> + </element> + </optional> + </define> + <define name="memorydev"> <element name="memory"> <attribute name="model"> @@ -4786,6 +4802,7 @@ <optional> <ref name="alias"/> </optional> + <ref name="hotplug"/> <optional> <ref name="address"/> </optional> diff --git a/src/conf/device_conf.h b/src/conf/device_conf.h index 8443de6..236bc96 100644 --- a/src/conf/device_conf.h +++ b/src/conf/device_conf.h @@ -142,6 +142,21 @@ typedef struct _virDomainDeviceInfo { /* bootIndex is only used for disk, network interface, hostdev * and redirdev devices */ unsigned int bootIndex; + /* hotpluggable values: + + * "no" - there is definitely no need for this device to be in a + * slot that supports hotplug. + * + * "yes" - assume hotplugging is required and refuse to put in any + * slot that doesn't support hotplug. (if the device + * itself doesn't support hotplug, that is an error) + * + * unspecified - the device can *manually* be placed in a + * non-hotpluggable slot, but that won't be automatically + * done (unless the device itself doesn't support + * hotplug). + */ + int hotpluggable; /* virTristateBool */ } virDomainDeviceInfo, *virDomainDeviceInfoPtr; diff --git a/src/conf/domain_addr.c b/src/conf/domain_addr.c index 5533b11..8f2a78f 100644 --- a/src/conf/domain_addr.c +++ b/src/conf/domain_addr.c @@ -709,6 +709,13 @@ virDomainPCIAddressReserveNextSlot(virDomainPCIAddressSetPtr addrs, virDomainPCIConnectFlags flags) { virPCIDeviceAddress addr; + + /* if config says to no require hotplug, then don't check for + * it when looking for an open address. + */ + if (dev->hotpluggable == VIR_TRISTATE_BOOL_NO) + flags &= ~VIR_PCI_CONNECT_HOTPLUGGABLE; + if (virDomainPCIAddressGetNextSlot(addrs, &addr, flags) < 0) return -1; diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c index 82876f3..8ebbca9 100644 --- a/src/conf/domain_conf.c +++ b/src/conf/domain_conf.c @@ -4765,6 +4765,18 @@ virDomainDeviceInfoFormat(virBufferPtr buf, virBufferAddLit(buf, "/>\n"); } + if (info->hotpluggable) { + const char *require = virTristateBoolTypeToString(info->hotpluggable); + + if (!require) { + virReportError(VIR_ERR_INTERNAL_ERROR, + _("unexpected hotplug require value %d"), + info->hotpluggable); + return -1; + } + virBufferAsprintf(buf, "<hotplug require='%s'/>\n", require); + } + if (info->type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_NONE || info->type == VIR_DOMAIN_DEVICE_ADDRESS_TYPE_VIRTIO_S390) return 0; @@ -5277,6 +5289,7 @@ virDomainDeviceInfoParseXML(xmlNodePtr node, xmlNodePtr alias = NULL; xmlNodePtr boot = NULL; xmlNodePtr rom = NULL; + xmlNodePtr hotplug = NULL; char *type = NULL; int ret = -1; @@ -5303,6 +5316,9 @@ virDomainDeviceInfoParseXML(xmlNodePtr node, (flags & VIR_DOMAIN_DEF_PARSE_ALLOW_ROM) && xmlStrEqual(cur->name, BAD_CAST "rom")) { rom = cur; + } else if (hotplug == NULL && + xmlStrEqual(cur->name, BAD_CAST "hotplug")) { + hotplug = cur; } } cur = cur->next; @@ -5335,6 +5351,18 @@ virDomainDeviceInfoParseXML(xmlNodePtr node, info->romfile = virXMLPropString(rom, "file"); } + if (hotplug) { + char *require = virXMLPropString(hotplug, "require"); + if (require && + ((info->hotpluggable = virTristateBoolTypeFromString(require)) <= 0)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("unknown hotplug require value '%s'"), require); + VIR_FREE(require); + goto cleanup; + } + VIR_FREE(require); + } + if (!address) return 0; diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-hotpluggable.args b/tests/qemuxml2argvdata/qemuxml2argv-q35-hotpluggable.args new file mode 100644 index 0000000..0020bfe --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-hotpluggable.args @@ -0,0 +1,50 @@ +LC_ALL=C \ +PATH=/bin \ +HOME=/home/test \ +USER=test \ +LOGNAME=test \ +QEMU_AUDIO_DRV=none \ +/usr/libexec/qemu-kvm \ +-name q35-test \ +-S \ +-M q35 \ +-m 2048 \ +-smp 2,sockets=2,cores=1,threads=1 \ +-uuid 11dbdcdd-4c3b-482b-8903-9bdb8c0a2774 \ +-nographic \ +-nodefaults \ +-monitor unix:/tmp/lib/domain--1-q35-test/monitor.sock,server,nowait \ +-no-acpi \ +-boot c \ +-device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e \ +-device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x0 \ +-device ich9-usb-ehci1,id=usb,bus=pcie.0,addr=0x1d.0x7 \ +-device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pcie.0,multifunction=on,\ +addr=0x1d \ +-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1 \ +-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2 \ +-drive file=/dev/HostVG/QEMUGuest1,format=raw,if=none,id=drive-sata0-0-0 \ +-device ide-drive,bus=ide.0,drive=drive-sata0-0-0,id=sata0-0-0 \ +-drive file=/dev/HostVG/QEMUGuest2,format=raw,if=none,id=drive-virtio-disk1 \ +-device virtio-blk-pci,bus=pci.2,addr=0x3,drive=drive-virtio-disk1,\ +id=virtio-disk1 \ +-drive file=/dev/HostVG/QEMUGuest2,format=raw,if=none,id=drive-virtio-disk2 \ +-device virtio-blk-pci,bus=pcie.0,addr=0x4,drive=drive-virtio-disk2,\ +id=virtio-disk2 \ +-device rtl8139,vlan=0,id=net0,mac=00:11:22:33:44:55,bus=pci.2,addr=0x1 \ +-net user,vlan=0,name=hostnet0 \ +-device rtl8139,vlan=1,id=net1,mac=00:11:22:33:44:55,bus=pcie.0,addr=0x2 \ +-net user,vlan=1,name=hostnet1 \ +-device virtio-net-pci,vlan=2,id=net2,mac=00:11:22:33:44:55,bus=pci.2,addr=0x2 \ +-net user,vlan=2,name=hostnet2 \ +-device virtio-net-pci,vlan=3,id=net3,mac=00:11:22:33:44:55,bus=pcie.0,\ +addr=0x3 \ +-net user,vlan=3,name=hostnet3 \ +-vga qxl \ +-global qxl-vga.ram_size=67108864 \ +-global qxl-vga.vram_size=33554432 \ +-device virtio-balloon-pci,id=balloon0,bus=pci.2,addr=0x4 \ +-object rng-random,id=objrng0,filename=/dev/random \ +-device virtio-rng-pci,rng=objrng0,id=rng0,bus=pcie.0,addr=0x5 \ +-object rng-random,id=objrng1,filename=/dev/random \ +-device virtio-rng-pci,rng=objrng1,id=rng1,bus=pci.2,addr=0x5 diff --git a/tests/qemuxml2argvdata/qemuxml2argv-q35-hotpluggable.xml b/tests/qemuxml2argvdata/qemuxml2argv-q35-hotpluggable.xml new file mode 100644 index 0000000..5c826ba --- /dev/null +++ b/tests/qemuxml2argvdata/qemuxml2argv-q35-hotpluggable.xml @@ -0,0 +1,62 @@ +<domain type='qemu'> + <name>q35-test</name> + <uuid>11dbdcdd-4c3b-482b-8903-9bdb8c0a2774</uuid> + <memory unit='KiB'>2097152</memory> + <currentMemory unit='KiB'>2097152</currentMemory> + <vcpu placement='static' cpuset='0-1'>2</vcpu> + <os> + <type arch='x86_64' machine='q35'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/libexec/qemu-kvm</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='sda' bus='sata'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest2'/> + <target dev='vdb' bus='virtio'/> + </disk> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest2'/> + <target dev='vdc' bus='virtio'/> + <hotplug require='no'/> + </disk> + <video> + <model type='qxl' ram='65536' vram='32768' vgamem='8192' heads='1'/> + </video> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='rtl8139'/> + </interface> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='rtl8139'/> + <hotplug require='no'/> + </interface> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='virtio'/> + </interface> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='virtio'/> + <hotplug require='no'/> + </interface> + <rng model='virtio'> + <hotplug require='no'/> + <backend model='random'>/dev/random</backend> + </rng> + <rng model='virtio'> + <hotplug require='yes'/> + <backend model='random'>/dev/random</backend> + </rng> + <memballoon model='virtio'/> + </devices> +</domain> diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 2de4f0a..2be634c 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -1677,6 +1677,14 @@ mymain(void) QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, QEMU_CAPS_VGA_QXL, QEMU_CAPS_DEVICE_QXL); + DO_TEST("q35-hotpluggable", + QEMU_CAPS_DEVICE_PCI_BRIDGE, + QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_ICH9_AHCI, + QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, + QEMU_CAPS_DEVICE_VIDEO_PRIMARY, + QEMU_CAPS_VGA_QXL, QEMU_CAPS_DEVICE_QXL, + QEMU_CAPS_DEVICE_VIRTIO_RNG, QEMU_CAPS_OBJECT_RNG_RANDOM); DO_TEST("pcie-root-port", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, diff --git a/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-hotpluggable.xml b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-hotpluggable.xml new file mode 100644 index 0000000..e771125 --- /dev/null +++ b/tests/qemuxml2xmloutdata/qemuxml2xmlout-q35-hotpluggable.xml @@ -0,0 +1,103 @@ +<domain type='qemu'> + <name>q35-test</name> + <uuid>11dbdcdd-4c3b-482b-8903-9bdb8c0a2774</uuid> + <memory unit='KiB'>2097152</memory> + <currentMemory unit='KiB'>2097152</currentMemory> + <vcpu placement='static' cpuset='0-1'>2</vcpu> + <os> + <type arch='x86_64' machine='q35'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/libexec/qemu-kvm</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='sda' bus='sata'/> + <address type='drive' controller='0' bus='0' target='0' unit='0'/> + </disk> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest2'/> + <target dev='vdb' bus='virtio'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x03' function='0x0'/> + </disk> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest2'/> + <target dev='vdc' bus='virtio'/> + <hotplug require='no'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> + </disk> + <controller type='usb' index='0' model='ich9-ehci1'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x7'/> + </controller> + <controller type='usb' index='0' model='ich9-uhci1'> + <master startport='0'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x0' multifunction='on'/> + </controller> + <controller type='usb' index='0' model='ich9-uhci2'> + <master startport='2'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x1'/> + </controller> + <controller type='usb' index='0' model='ich9-uhci3'> + <master startport='4'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1d' function='0x2'/> + </controller> + <controller type='sata' index='0'> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> + </controller> + <controller type='pci' index='0' model='pcie-root'/> + <controller type='pci' index='1' model='dmi-to-pci-bridge'> + <model name='i82801b11-bridge'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x1e' function='0x0'/> + </controller> + <controller type='pci' index='2' model='pci-bridge'> + <model name='pci-bridge'/> + <target chassisNr='2'/> + <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> + </controller> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='rtl8139'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/> + </interface> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='rtl8139'/> + <hotplug require='no'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> + </interface> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='virtio'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x02' function='0x0'/> + </interface> + <interface type='user'> + <mac address='00:11:22:33:44:55'/> + <model type='virtio'/> + <hotplug require='no'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> + </interface> + <input type='mouse' bus='ps2'/> + <input type='keyboard' bus='ps2'/> + <video> + <model type='qxl' ram='65536' vram='32768' vgamem='8192' heads='1' primary='yes'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> + </video> + <memballoon model='virtio'> + <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/> + </memballoon> + <rng model='virtio'> + <backend model='random'>/dev/random</backend> + <hotplug require='no'/> + <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> + </rng> + <rng model='virtio'> + <backend model='random'>/dev/random</backend> + <hotplug require='yes'/> + <address type='pci' domain='0x0000' bus='0x02' slot='0x05' function='0x0'/> + </rng> + </devices> +</domain> diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c index 7601a5f..2d8145b 100644 --- a/tests/qemuxml2xmltest.c +++ b/tests/qemuxml2xmltest.c @@ -691,6 +691,11 @@ mymain(void) QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, QEMU_CAPS_VGA_QXL, QEMU_CAPS_DEVICE_QXL); + DO_TEST("q35-hotpluggable", + QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, + QEMU_CAPS_ICH9_AHCI, QEMU_CAPS_PCI_MULTIFUNCTION, + QEMU_CAPS_ICH9_USB_EHCI1, QEMU_CAPS_DEVICE_VIDEO_PRIMARY, + QEMU_CAPS_VGA_QXL, QEMU_CAPS_DEVICE_QXL); DO_TEST("pcie-root", QEMU_CAPS_DEVICE_PCI_BRIDGE, QEMU_CAPS_DEVICE_DMI_TO_PCI_BRIDGE, -- 2.7.4 -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list