Hi Eric, > -----Original Message----- > From: Eric Auger [mailto:eric.auger@xxxxxxxxxx] > Sent: 30 September 2021 10:48 > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@xxxxxxxxxx>; > linux-arm-kernel@xxxxxxxxxxxxxxxxxxx; linux-acpi@xxxxxxxxxxxxxxx; > iommu@xxxxxxxxxxxxxxxxxxxxxxxxxx; Jean-Philippe Brucker > <Jean-Philippe.Brucker@xxxxxxx> > Cc: Linuxarm <linuxarm@xxxxxxxxxx>; lorenzo.pieralisi@xxxxxxx; > joro@xxxxxxxxxx; robin.murphy@xxxxxxx; will@xxxxxxxxxx; wanghuiqiang > <wanghuiqiang@xxxxxxxxxx>; Guohanjun (Hanjun Guo) > <guohanjun@xxxxxxxxxx>; steven.price@xxxxxxx; Sami.Mujawar@xxxxxxx; > jon@xxxxxxxxxxxxx; yangyicong <yangyicong@xxxxxxxxxx> > Subject: Re: [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node > > Hi Shameer, > > On 8/5/21 10:07 AM, Shameer Kolothum wrote: > > Hi, > > > > The series adds support to IORT RMR nodes specified in IORT > > Revision E.b -ARM DEN 0049E[0]. RMR nodes are used to describe > > memory ranges that are used by endpoints and require a unity > > mapping in SMMU. > > I used your series and RMRs to force a guest iommu (vSMMUv3 nested stage > use case) to have a flat mapping for IOVAs within [0x8000000, 0x8100000] > (matching MSI_IOVA_BASE and MSI_IOVA_LENGTH) used by the host to map > MSI > physical doorbells. > > That way when an assigned device protected by a vSMMUv3 implemented > upon > nested stage issues an MSI transaction, let's say using IOVA=0x8000000, > we would get: > S1 (guest) S2 (host) > 0x8000000 0x8000000 Physical DB > > This method was suggested by Jean-Philippe (added in CC) and it > simplifies the nested stage integration because we don't have to care > about nested stage MSI bindings. > > However if I understand correctly we cannot define a range of SIDs using > the same RMR (due to the single mapping bit which must be set, Table 5 > flags format). This is a spec restriction and not an issue with your series. Yes. The spec currently mandates single mapping bit to be set. > > As VFIO devices can be hot-plugged we thus need to create as many RMR > nodes as potential BDFs, leading to 256 * 6 = 1536 RMR nodes if you have > 5 pcie root ports as it is usual in VMs. Then this causes some trouble > at qemu level for instance, wrt migration. See [RFC] > hw/arm/virt-acpi-build: Add IORT RMR regions to handle MSI nested binding. > > Do you know if there is a plan to remove the single mapping limitation > in the spec? I would imagine so. In an earlier comment[1], Robin did mention about possible relaxing of this in future spec revision. Thanks, Shameer 1. https://lore.kernel.org/linux-arm-kernel/15c7fac0-11a8-0cdb-aac3-b5d8feb8f066@xxxxxxx/ > Thanks > > Eric > > > > We have faced issues with 3408iMR RAID controller cards which > > fail to boot when SMMU is enabled. This is because these > > controllers make use of host memory for various caching related > > purposes and when SMMU is enabled the iMR firmware fails to > > access these memory regions as there is no mapping for them. > > IORT RMR provides a way for UEFI to describe and report these > > memory regions so that the kernel can make a unity mapping for > > these in SMMU. > > > > Change History: > > > > v6 --> v7 > > > > The only change from v6 is the fix pointed out by Steve to > > the SMMUv2 SMR bypass install in patch #8. > > > > Thanks to the Tested-by tags by Laurentiu with SMMUv2 and > > Hanjun/Huiqiang with SMMUv3 for v6. I haven't added the tags > > yet as the series still needs more review[1]. > > > > Feedback and tests on this series is very much appreciated. > > > > v5 --> v6 > > - Addressed comments from Robin & Lorenzo. > > : Moved iort_parse_rmr() to acpi_iort_init() from > > iort_init_platform_devices(). > > : Removed use of struct iort_rmr_entry during the initial > > parse. Using struct iommu_resv_region instead. > > : Report RMR address alignment and overlap errors, but continue. > > : Reworked arm_smmu_init_bypass_stes() (patch # 6). > > - Updated SMMUv2 bypass SMR code. Thanks to Jon N (patch #8). > > - Set IOMMU protection flags(IOMMU_CACHE, IOMMU_MMIO) based > > on Type of RMR region. Suggested by Jon N. > > > > Thanks, > > Shameer > > [0] https://developer.arm.com/documentation/den0049/latest/ > > [1] > https://lore.kernel.org/linux-acpi/20210716083442.1708-1-shameerali.koloth > um.thodi@xxxxxxxxxx/T/#m043c95b869973a834b2fd57f3e1ed0325c84f3b7 > > ------ > > v4 --> v5 > > -Added a fw_data union to struct iommu_resv_region and removed > > struct iommu_rmr (Based on comments from Joerg/Robin). > > -Added iommu_put_rmrs() to release mem. > > -Thanks to Steve for verifying on SMMUv2, but not added the Tested-by > > yet because of the above changes. > > > > v3 -->v4 > > -Included the SMMUv2 SMR bypass install changes suggested by > > Steve(patch #7) > > -As per Robin's comments, RMR reserve implementation is now > > more generic (patch #8) and dropped v3 patches 8 and 10. > > -Rebase to 5.13-rc1 > > > > RFC v2 --> v3 > > -Dropped RFC tag as the ACPICA header changes are now ready to be > > part of 5.13[0]. But this series still has a dependency on that patch. > > -Added IORT E.b related changes(node flags, _DSM function 5 checks for > > PCIe). > > -Changed RMR to stream id mapping from M:N to M:1 as per the spec and > > discussion here[1]. > > -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!) > > ------ > > > > Jon Nettleton (1): > > iommu/arm-smmu: Get associated RMR info and install bypass SMR > > > > Shameer Kolothum (8): > > iommu: Introduce a union to struct iommu_resv_region > > ACPI/IORT: Add support for RMR node parsing > > iommu/dma: Introduce generic helper to retrieve RMR info > > ACPI/IORT: Add a helper to retrieve RMR memory regions > > iommu/arm-smmu-v3: Introduce strtab init helper > > iommu/arm-smmu-v3: Refactor arm_smmu_init_bypass_stes() to force > > bypass > > iommu/arm-smmu-v3: Get associated RMR info and install bypass STE > > iommu/dma: Reserve any RMR regions associated with a dev > > > > drivers/acpi/arm64/iort.c | 172 > +++++++++++++++++++- > > drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 76 +++++++-- > > drivers/iommu/arm/arm-smmu/arm-smmu.c | 48 ++++++ > > drivers/iommu/dma-iommu.c | 89 +++++++++- > > include/linux/acpi_iort.h | 7 + > > include/linux/dma-iommu.h | 13 ++ > > include/linux/iommu.h | 11 ++ > > 7 files changed, 393 insertions(+), 23 deletions(-) > >
