On Thu, Jan 30, 2025 at 10:00:21AM -0800, Nathan Chen wrote: > > > > Add a pcie-expander-bus controller to the VM definition for each "nestedSmmuv3" > > > device that is generated when the "nestedSmmuv3" IOMMU model is parsed from the > > > VM definition. Assign each "nestedSmmuv3" device to one PXB controller, and > > > route any unmanaged "hostdev" VFIO devices with associated host SMMU nodes to > > > their corresponding PXB controller based on the "name" attributes of > > > "nestedSmmuv3" devices attached to these PXB controllers. > > > > > > Signed-off-by: Nathan Chen<nathanc@xxxxxxxxxx> > > > --- > > > src/conf/domain_addr.c | 26 ++++++- > > > src/conf/domain_addr.h | 3 +- > > > src/conf/domain_conf.c | 1 + > > > src/qemu/qemu_domain_address.c | 134 +++++++++++++++++++++++++++++++++ > > > 4 files changed, 162 insertions(+), 2 deletions(-) > > > diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c > > > index 31004bfc7e..dee198a7d2 100644 > > > --- a/src/qemu/qemu_domain_address.c > > > +++ b/src/qemu/qemu_domain_address.c > > > @@ -1627,6 +1627,19 @@ qemuDomainPCIAddressSetCreate(virDomainDef *def, > > > addrs->dryRun = dryRun; > > > + /* PXB indices must come before pcie-root-port indices in qemu, > > > + * so add PXB buses to addrs before the pcie-root-ports. */ > > Can you elaborate on that requirement - I don't get why QEMU cares > > which order slots are chosen in for PXB vs PCIe-Root-Port devices. > > The libvirt controller index describes the order in which bus controllers > are encountered; if the pcie-root-port index is less than the PXB index, the > pcie-root-port will not find a PXB to attach to and VM creation will fail. > To address this, PXBs must be added to the VM definition first and get > assigned a lower index than the pcie-root-ports. Oh, so you're specifically refering to a case of a pcie-root-port that is attached to a PXB that is listed 2nd in the XML, rather than relative to all possible PXBs. Assigning PXB indexes first fixes the problem in cases where libvirt is the one assigning indexes, but if users manually assign any indexes the problem could still hit. Possibly libvirt needs to be more intelligent about the order in which it specifies the controller devices on the QEMU command line, rather than blindly following XML or index order With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
