> -----Original Message----- > From: Nicolin Chen <nicolinc@xxxxxxxxxx> > Sent: Wednesday, August 28, 2024 7:13 PM > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@xxxxxxxxxx> > Cc: Jason Gunthorpe <jgg@xxxxxxxxxx>; acpica-devel@xxxxxxxxxxxxxxx; > Guohanjun (Hanjun Guo) <guohanjun@xxxxxxxxxx>; > iommu@xxxxxxxxxxxxxxx; Joerg Roedel <joro@xxxxxxxxxx>; Kevin Tian > <kevin.tian@xxxxxxxxx>; kvm@xxxxxxxxxxxxxxx; Len Brown > <lenb@xxxxxxxxxx>; linux-acpi@xxxxxxxxxxxxxxx; linux-arm- > kernel@xxxxxxxxxxxxxxxxxxx; Lorenzo Pieralisi <lpieralisi@xxxxxxxxxx>; Rafael J. > Wysocki <rafael@xxxxxxxxxx>; Robert Moore <robert.moore@xxxxxxxxx>; > Robin Murphy <robin.murphy@xxxxxxx>; Sudeep Holla > <sudeep.holla@xxxxxxx>; Will Deacon <will@xxxxxxxxxx>; Alex Williamson > <alex.williamson@xxxxxxxxxx>; Eric Auger <eric.auger@xxxxxxxxxx>; Jean- > Philippe Brucker <jean-philippe@xxxxxxxxxx>; Moritz Fischer > <mdf@xxxxxxxxxx>; Michael Shavit <mshavit@xxxxxxxxxx>; > patches@xxxxxxxxxxxxxxx; Mostafa Saleh <smostafa@xxxxxxxxxx> > Subject: Re: [PATCH v2 0/8] Initial support for SMMUv3 nested translation > > On Wed, Aug 28, 2024 at 06:06:36PM +0000, Shameerali Kolothum Thodi > wrote: > > > > > As mentioned above, the VIOMMU series would be required to test > the > > > > > entire nesting feature, which now has a v2 rebasing on this series. > > > > > I tested it with a paring QEMU branch. Please refer to: > > > > > https://lore.kernel.org/linux- > > > > > iommu/cover.1724776335.git.nicolinc@xxxxxxxxxx/ > > > > > > > > Thanks for this. I haven't gone through the viommu and its Qemu > branch > > > > yet. The way we present nested-smmuv3/iommufd to the Qemu seems > to > > > > have changed with the above Qemu branch(multiple nested SMMUs). > > > > The old Qemu command line for nested setup doesn't work anymore. > > > > > > > > Could you please share an example Qemu command line to verify this > > > > series(Sorry, if I missed it in the links/git). > > > > > > My bad. I updated those two "for_iommufd_" QEMU branches with a > > > README commit on top of each for the reference command. > > > > Thanks. I did give it a go and this is my command line based on above, > > > But it fails to boot very early: > > > > root@ubuntu:/home/shameer/qemu-test# ./qemu_run-simple-iommufd- > nicolin-2 > > qemu-system-aarch64-nicolin-viommu: Illegal numa node 2 > > > > Any idea what am I missing? Do you any special config enabled while > building Qemu? > > Looks like you are running on a multi-SMMU platform :) > > Would you please try syncing your local branch? That should work, > as the update also had a small change to the virt code: > > diff --git a/hw/arm/virt.c b/hw/arm/virt.c > index 161a28a311..a782909016 100644 > --- a/hw/arm/virt.c > +++ b/hw/arm/virt.c > @@ -1640,7 +1640,7 @@ static PCIBus > *create_pcie_expander_bridge(VirtMachineState *vms, uint8_t idx) > } > > qdev_prop_set_uint8(dev, "bus_nr", bus_nr); > - qdev_prop_set_uint16(dev, "numa_node", idx); > + qdev_prop_set_uint16(dev, "numa_node", 0); > qdev_realize_and_unref(dev, BUS(bus), &error_fatal); That makes some progress. But still I am not seeing the assigned dev in Guest. -device vfio-pci-nohotplug,host=0000:75:00.1,iommufd=iommufd0 root@ubuntu:/# lspci -tv# root@ubuntu:/# lspci -tv -+-[0000:ca]---00.0-[cb]-- \-[0000:00]-+-00.0 Red Hat, Inc. QEMU PCIe Host bridge +-01.0 Red Hat, Inc Virtio network device +-02.0 Red Hat, Inc. QEMU PCIe Expander bridge +-03.0 Red Hat, Inc. QEMU PCIe Expander bridge +-04.0 Red Hat, Inc. QEMU PCIe Expander bridge +-05.0 Red Hat, Inc. QEMU PCIe Expander bridge +-06.0 Red Hat, Inc. QEMU PCIe Expander bridge +-07.0 Red Hat, Inc. QEMU PCIe Expander bridge +-08.0 Red Hat, Inc. QEMU PCIe Expander bridge \-09.0 Red Hat, Inc. QEMU PCIe Expander bridge The new root port is created, but no device attached. But without iommufd, -device vfio-pci-nohotplug,host=0000:75:00.1 root@ubuntu:/# lspci -tv -[0000:00]-+-00.0 Red Hat, Inc. QEMU PCIe Host bridge +-01.0 Red Hat, Inc Virtio network device +-02.0 Red Hat, Inc. QEMU PCIe Expander bridge +-03.0 Red Hat, Inc. QEMU PCIe Expander bridge +-04.0 Red Hat, Inc. QEMU PCIe Expander bridge +-05.0 Red Hat, Inc. QEMU PCIe Expander bridge +-06.0 Red Hat, Inc. QEMU PCIe Expander bridge +-07.0 Red Hat, Inc. QEMU PCIe Expander bridge +-08.0 Red Hat, Inc. QEMU PCIe Expander bridge +-09.0 Red Hat, Inc. QEMU PCIe Expander bridge \-0a.0 Huawei Technologies Co., Ltd. Device a251 We can see dev a251. And yes the setup has multiple SMMUs(8). Thanks, Shameer
