> -----Original Message----- > From: Nicolin Chen <nicolinc@xxxxxxxxxx> > Sent: Thursday, August 29, 2024 5:10 PM > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@xxxxxxxxxx> > Cc: Jason Gunthorpe <jgg@xxxxxxxxxx>; acpica-devel@xxxxxxxxxxxxxxx; > Guohanjun (Hanjun Guo) <guohanjun@xxxxxxxxxx>; iommu@xxxxxxxxxxxxxxx; > Joerg Roedel <joro@xxxxxxxxxx>; Kevin Tian <kevin.tian@xxxxxxxxx>; > kvm@xxxxxxxxxxxxxxx; Len Brown <lenb@xxxxxxxxxx>; linux- > acpi@xxxxxxxxxxxxxxx; linux-arm-kernel@xxxxxxxxxxxxxxxxxxx; Lorenzo Pieralisi > <lpieralisi@xxxxxxxxxx>; Rafael J. Wysocki <rafael@xxxxxxxxxx>; Robert Moore > <robert.moore@xxxxxxxxx>; Robin Murphy <robin.murphy@xxxxxxx>; Sudeep > Holla <sudeep.holla@xxxxxxx>; Will Deacon <will@xxxxxxxxxx>; Alex > Williamson <alex.williamson@xxxxxxxxxx>; Eric Auger > <eric.auger@xxxxxxxxxx>; Jean-Philippe Brucker <jean-philippe@xxxxxxxxxx>; > Moritz Fischer <mdf@xxxxxxxxxx>; Michael Shavit <mshavit@xxxxxxxxxx>; > patches@xxxxxxxxxxxxxxx; Mostafa Saleh <smostafa@xxxxxxxxxx> > Subject: Re: [PATCH v2 0/8] Initial support for SMMUv3 nested translation > > On Thu, Aug 29, 2024 at 02:52:23PM +0000, Shameerali Kolothum Thodi wrote: > > > That makes some progress. But still I am not seeing the assigned dev in > > > Guest. > > > > > > -device vfio-pci-nohotplug,host=0000:75:00.1,iommufd=iommufd0 > > > > > > root@ubuntu:/# lspci -tv# > > > > > > root@ubuntu:/# lspci -tv > > > -+-[0000:ca]---00.0-[cb]-- > > > \-[0000:00]-+-00.0 Red Hat, Inc. QEMU PCIe Host bridge > > > +-01.0 Red Hat, Inc Virtio network device > > > +-02.0 Red Hat, Inc. QEMU PCIe Expander bridge > > > +-03.0 Red Hat, Inc. QEMU PCIe Expander bridge > > > +-04.0 Red Hat, Inc. QEMU PCIe Expander bridge > > > +-05.0 Red Hat, Inc. QEMU PCIe Expander bridge > > > +-06.0 Red Hat, Inc. QEMU PCIe Expander bridge > > > +-07.0 Red Hat, Inc. QEMU PCIe Expander bridge > > > +-08.0 Red Hat, Inc. QEMU PCIe Expander bridge > > > \-09.0 Red Hat, Inc. QEMU PCIe Expander bridge > > Hmm, the tree looks correct.. > > > > The new root port is created, but no device attached. > > It looks like Guest finds the config invalid: > > > > [ 0.283618] PCI host bridge to bus 0000:ca > > [ 0.284064] ACPI BIOS Error (bug): > \_SB.PCF7.PCEE.PCE5.PCDC.PCD3.PCCA._DSM: Excess arguments - ASL declared > 5, ACPI requires 4 (20240322/nsarguments-162) > > Looks like the DSM change wasn't clean. Yet, this might not be the > root cause, as mine could boot with it. Yes. This is not the culprit in this case and was reported earlier as well, https://patchew.org/QEMU/20211005085313.493858-1-eric.auger@xxxxxxxxxx/20211005085313.493858-2-eric.auger@xxxxxxxxxx/ > Here is mine (I added a print to that conflict part, for success): > > [ 0.340733] ACPI BIOS Error (bug): \_SB.PCF7.PCEE.PCE5.PCDC._DSM: Excess > arguments - ASL declared 5, ACPI requires 4 (20230628/nsarguments-162) > [ 0.341776] pci 0000:dc:00.0: [1b36:000c] type 01 class 0x060400 PCIe Root > Port > [ 0.344895] pci 0000:dc:00.0: BAR 0 [mem 0x10400000-0x10400fff] > [ 0.347935] pci 0000:dc:00.0: PCI bridge to [bus dd] > [ 0.348410] pci 0000:dc:00.0: bridge window [mem 0x10200000-0x103fffff] > [ 0.349483] pci 0000:dc:00.0: bridge window [mem 0x42000000000- > 0x44080ffffff 64bit pref] > [ 0.351459] pci_bus 0000:dd: busn_res: insert [bus dd] under [bus dc-dd] > > In my case: > [root bus (00)] <---[pxb (dc)] <--- [root-port (dd)] <--- dev > > In your case: > [root bus (00)] <---[pxb (ca)] <--- [root-port (cb)] <--- dev > > > [ 0.285533] pci_bus 0000:ca: root bus resource [bus ca] > > [ 0.286214] pci 0000:ca:00.0: [1b36:000c] type 01 class 0x060400 PCIe Root > Port > > [ 0.287717] pci 0000:ca:00.0: BAR 0 [mem 0x00000000-0x00000fff] > > [ 0.288431] pci 0000:ca:00.0: PCI bridge to [bus 00] > > This starts to diff. Somehow the link is reversed? It should be: > [ 0.288431] pci 0000:ca:00.0: PCI bridge to [bus cb] > > > [ 0.290649] pci 0000:ca:00.0: bridge configuration invalid ([bus 00-00]), > reconfiguring > > [ 0.292476] pci_bus 0000:cb: busn_res: can not insert [bus cb-ca] under [bus > ca] (conflicts with (null) [bus ca]) > > [ 0.293597] pci_bus 0000:cb: busn_res: [bus cb-ca] end is updated to cb > > [ 0.294300] pci_bus 0000:cb: busn_res: can not insert [bus cb] under [bus ca] > (conflicts with (null) [bus ca]) > > And then everything went south... > > Would you please try adding some prints? > ---------------------------------------------------------------------- > @@ -1556,6 +1556,7 @@ static char *create_new_pcie_port(VirtNestedSmmu > *nested_smmu, Error **errp) > uint32_t bus_nr = pci_bus_num(nested_smmu->pci_bus); > DeviceState *dev; > char *name_port; > + bool ret; > > /* Create a root port */ > dev = qdev_new("pcie-root-port"); > @@ -1571,7 +1572,9 @@ static char *create_new_pcie_port(VirtNestedSmmu > *nested_smmu, Error **errp) > qdev_prop_set_uint32(dev, "chassis", chassis_nr); > qdev_prop_set_uint32(dev, "slot", port_nr); > qdev_prop_set_uint64(dev, "io-reserve", 0); > - qdev_realize_and_unref(dev, BUS(nested_smmu->pci_bus), &error_fatal); > + ret = qdev_realize_and_unref(dev, BUS(nested_smmu->pci_bus), > &error_fatal); > + fprintf(stderr, "ret=%d, pcie-root-port ID: %s, added to pxb_bus num: %x, > chassis: %d\n", > + ret, name_port, pci_bus_num(nested_smmu->pci_bus), chassis_nr); > return name_port; > } Print shows everything fine: create_new_pcie_port: name_port smmu_bus0xca_port0, bus_nr 0xca chassis_nr 0xfd, nested_smmu->index 0x2, pci_bus_num 0xca, ret 1 It looks like a problem with old QEMU_EFI.fd(2022 build and before). I tried with 2023 QEMU_EFI.fd and with that it looks fine. root@ubuntu:/# lspci -tv -+-[0000:ca]---00.0-[cb]----00.0 Huawei Technologies Co., Ltd. Device a251 \-[0000:00]-+-00.0 Red Hat, Inc. QEMU PCIe Host bridge +-01.0 Red Hat, Inc Virtio network device +-02.0 Red Hat, Inc. QEMU PCIe Expander bridge +-03.0 Red Hat, Inc. QEMU PCIe Expander bridge +-04.0 Red Hat, Inc. QEMU PCIe Expander bridge +-05.0 Red Hat, Inc. QEMU PCIe Expander bridge +-06.0 Red Hat, Inc. QEMU PCIe Expander bridge +-07.0 Red Hat, Inc. QEMU PCIe Expander bridge +-08.0 Red Hat, Inc. QEMU PCIe Expander bridge \-09.0 Red Hat, Inc. QEMU PCIe Expander bridge So for now, I can proceed. Thanks, Shameer
