On Thu, Aug 29, 2024 at 02:52:23PM +0000, Shameerali Kolothum Thodi wrote:
> > That makes some progress. But still I am not seeing the assigned dev in
> > Guest.
> >
> > -device vfio-pci-nohotplug,host=0000:75:00.1,iommufd=iommufd0
> >
> > root@ubuntu:/# lspci -tv#
> >
> > root@ubuntu:/# lspci -tv
> > -+-[0000:ca]---00.0-[cb]--
> > \-[0000:00]-+-00.0 Red Hat, Inc. QEMU PCIe Host bridge
> > +-01.0 Red Hat, Inc Virtio network device
> > +-02.0 Red Hat, Inc. QEMU PCIe Expander bridge
> > +-03.0 Red Hat, Inc. QEMU PCIe Expander bridge
> > +-04.0 Red Hat, Inc. QEMU PCIe Expander bridge
> > +-05.0 Red Hat, Inc. QEMU PCIe Expander bridge
> > +-06.0 Red Hat, Inc. QEMU PCIe Expander bridge
> > +-07.0 Red Hat, Inc. QEMU PCIe Expander bridge
> > +-08.0 Red Hat, Inc. QEMU PCIe Expander bridge
> > \-09.0 Red Hat, Inc. QEMU PCIe Expander bridge
Hmm, the tree looks correct..
> > The new root port is created, but no device attached.
> It looks like Guest finds the config invalid:
>
> [ 0.283618] PCI host bridge to bus 0000:ca
> [ 0.284064] ACPI BIOS Error (bug): \_SB.PCF7.PCEE.PCE5.PCDC.PCD3.PCCA._DSM: Excess arguments - ASL declared 5, ACPI requires 4 (20240322/nsarguments-162)
Looks like the DSM change wasn't clean. Yet, this might not be the
root cause, as mine could boot with it.
Here is mine (I added a print to that conflict part, for success):
[ 0.340733] ACPI BIOS Error (bug): \_SB.PCF7.PCEE.PCE5.PCDC._DSM: Excess arguments - ASL declared 5, ACPI requires 4 (20230628/nsarguments-162)
[ 0.341776] pci 0000:dc:00.0: [1b36:000c] type 01 class 0x060400 PCIe Root Port
[ 0.344895] pci 0000:dc:00.0: BAR 0 [mem 0x10400000-0x10400fff]
[ 0.347935] pci 0000:dc:00.0: PCI bridge to [bus dd]
[ 0.348410] pci 0000:dc:00.0: bridge window [mem 0x10200000-0x103fffff]
[ 0.349483] pci 0000:dc:00.0: bridge window [mem 0x42000000000-0x44080ffffff 64bit pref]
[ 0.351459] pci_bus 0000:dd: busn_res: insert [bus dd] under [bus dc-dd]
In my case:
[root bus (00)] <---[pxb (dc)] <--- [root-port (dd)] <--- dev
In your case:
[root bus (00)] <---[pxb (ca)] <--- [root-port (cb)] <--- dev
> [ 0.285533] pci_bus 0000:ca: root bus resource [bus ca]
> [ 0.286214] pci 0000:ca:00.0: [1b36:000c] type 01 class 0x060400 PCIe Root Port
> [ 0.287717] pci 0000:ca:00.0: BAR 0 [mem 0x00000000-0x00000fff]
> [ 0.288431] pci 0000:ca:00.0: PCI bridge to [bus 00]
This starts to diff. Somehow the link is reversed? It should be:
[ 0.288431] pci 0000:ca:00.0: PCI bridge to [bus cb]
> [ 0.290649] pci 0000:ca:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
> [ 0.292476] pci_bus 0000:cb: busn_res: can not insert [bus cb-ca] under [bus ca] (conflicts with (null) [bus ca])
> [ 0.293597] pci_bus 0000:cb: busn_res: [bus cb-ca] end is updated to cb
> [ 0.294300] pci_bus 0000:cb: busn_res: can not insert [bus cb] under [bus ca] (conflicts with (null) [bus ca])
And then everything went south...
Would you please try adding some prints?
----------------------------------------------------------------------
@@ -1556,6 +1556,7 @@ static char *create_new_pcie_port(VirtNestedSmmu *nested_smmu, Error **errp)
uint32_t bus_nr = pci_bus_num(nested_smmu->pci_bus);
DeviceState *dev;
char *name_port;
+ bool ret;
/* Create a root port */
dev = qdev_new("pcie-root-port");
@@ -1571,7 +1572,9 @@ static char *create_new_pcie_port(VirtNestedSmmu *nested_smmu, Error **errp)
qdev_prop_set_uint32(dev, "chassis", chassis_nr);
qdev_prop_set_uint32(dev, "slot", port_nr);
qdev_prop_set_uint64(dev, "io-reserve", 0);
- qdev_realize_and_unref(dev, BUS(nested_smmu->pci_bus), &error_fatal);
+ ret = qdev_realize_and_unref(dev, BUS(nested_smmu->pci_bus), &error_fatal);
+ fprintf(stderr, "ret=%d, pcie-root-port ID: %s, added to pxb_bus num: %x, chassis: %d\n",
+ ret, name_port, pci_bus_num(nested_smmu->pci_bus), chassis_nr);
return name_port;
}
----------------------------------------------------------------------
We should make sure that the 'bus_nr' and 'bus' are set correctly
and all the realize() returned true?:
Thanks
Nicolin
