> -----Original Message----- > From: Steven Price [mailto:steven.price@xxxxxxx] > Sent: 21 May 2021 13:55 > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@xxxxxxxxxx>; > linux-arm-kernel@xxxxxxxxxxxxxxxxxxx; linux-acpi@xxxxxxxxxxxxxxx; > iommu@xxxxxxxxxxxxxxxxxxxxxxxxxx > Cc: Linuxarm <linuxarm@xxxxxxxxxx>; lorenzo.pieralisi@xxxxxxx; > joro@xxxxxxxxxx; robin.murphy@xxxxxxx; wanghuiqiang > <wanghuiqiang@xxxxxxxxxx>; Guohanjun (Hanjun Guo) > <guohanjun@xxxxxxxxxx>; Sami.Mujawar@xxxxxxx; jon@xxxxxxxxxxxxx; > eric.auger@xxxxxxxxxx; yangyicong <yangyicong@xxxxxxxxxx> > Subject: Re: [PATCH v4 0/8] ACPI/IORT: Support for IORT RMR node > > On 13/05/2021 14:45, Shameer Kolothum wrote: > > Hi, > > > > v3 -->v4 > > -Included the SMMUv2 SMR bypass install changes suggested by > > Steve(patch #7) > > -As per Robin's comments, RMR reserve implementation is now > > more generic (patch #8) and dropped v3 patches 8 and 10. > > -Rebase to 5.13-rc1 > > > > The whole series is available here, > > https://github.com/hisilicon/kernel-dev/tree/private-v5.13-rc1-rmr-v4-ext > > > > RFC v2 --> v3 > > -Dropped RFC tag as the ACPICA header changes are now ready to be > > part of 5.13[0]. But this series still has a dependency on that patch. > > -Added IORT E.b related changes(node flags, _DSM function 5 checks for > > PCIe). > > -Changed RMR to stream id mapping from M:N to M:1 as per the spec and > > discussion here[1]. > > -Last two patches add support for SMMUv2(Thanks to Jon Nettleton!) > > > > Sanity tested on a HiSilicon D06. Further testing and feedback is greatly > > appreciated. > > With the updated SMMUv2 support this works fine on my Juno with EFIFB > (and corresponding patches to the firmware to expose the ACPI tables). > Feel free to add > > Tested-by: Steven Price <steven.price@xxxxxxx> Thanks Steve. I am in the process of incorporating the comments from Joerg/Robin to reuse the struct iommu_resv_region. I will post a v5 soon with that and a couple of other minor fixes. Thanks, Shameer > Thanks, > > Steve > > > https://github.com/hisilicon/kernel-dev/tree/private-v5.12-rc8-rmr-v3 > > > > Thanks, > > Shameer > > > > [0] > https://lore.kernel.org/linux-acpi/20210406213028.718796-22-erik.kaneda@i > ntel.com/ > > [1] > https://op-lists.linaro.org/pipermail/linaro-open-discussions/2021-April/00015 > 0.html > > > > RFC v1 --> v2: > > - Added a generic interface for IOMMU drivers to retrieve all the > > RMR info associated with a given IOMMU. > > - SMMUv3 driver gets the RMR list during probe() and installs > > bypass STEs for all the SIDs in the RMR list. This is to keep > > the ongoing traffic alive(if any) during SMMUv3 reset. This is > > based on the suggestions received for v1 to take care of the > > EFI framebuffer use case. Only sanity tested for now. > > - During the probe/attach device, SMMUv3 driver reserves any > > RMR region associated with the device such that there is a unity > > mapping for them in SMMU. > > --- > > > > From RFC v1: > > ------------- > > The series adds support to IORT RMR nodes specified in IORT > > Revision E -ARM DEN 0049E[0]. RMR nodes are used to describe memory > > ranges that are used by endpoints and require a unity mapping > > in SMMU. > > > > We have faced issues with 3408iMR RAID controller cards which > > fail to boot when SMMU is enabled. This is because these controllers > > make use of host memory for various caching related purposes and when > > SMMU is enabled the iMR firmware fails to access these memory regions > > as there is no mapping for them. IORT RMR provides a way for UEFI to > > describe and report these memory regions so that the kernel can make > > a unity mapping for these in SMMU. > > > > Tests: > > > > With a UEFI, that reports the RMR for the dev, > > .... > > [16F0h 5872 1] Type : 06 > > [16F1h 5873 2] Length : 007C > > [16F3h 5875 1] Revision : 00 > > [1038h 0056 2] Reserved : 00000000 > > [1038h 0056 2] Identifier : 00000000 > > [16F8h 5880 4] Mapping Count : 00000001 > > [16FCh 5884 4] Mapping Offset : 00000040 > > > > [1700h 5888 4] Number of RMR Descriptors : 00000002 > > [1704h 5892 4] RMR Descriptor Offset : 00000018 > > > > [1708h 5896 8] Base Address of RMR : 0000E6400000 > > [1710h 5904 8] Length of RMR : 000000100000 > > [1718h 5912 4] Reserved : 00000000 > > > > [171Ch 5916 8] Base Address of RMR : 0000000027B00000 > > [1724h 5924 8] Length of RMR : 0000000000C00000 > > [172Ch 5932 4] Reserved : 00000000 > > > > [1730h 5936 4] Input base : 00000000 > > [1734h 5940 4] ID Count : 00000001 > > [1738h 5944 4] Output Base : 00000003 > > [173Ch 5948 4] Output Reference : 00000064 > > [1740h 5952 4] Flags (decoded below) : 00000001 > > Single Mapping : 1 > > ... > > > > Without the series the RAID controller initialization fails as > > below, > > > > ... > > [ 12.631117] megaraid_sas 0000:03:00.0: FW supports sync > cache : Yes > > [ 12.637360] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is > called outbound_intr_mask:0x40000009 > > [ 18.776377] megaraid_sas 0000:03:00.0: Init cmd return status FAILED > for SCSI host 0 > > [ 23.019383] megaraid_sas 0000:03:00.0: Waiting for FW to come to > ready state > > [ 106.684281] megaraid_sas 0000:03:00.0: FW in FAULT state, Fault > code:0x10000 subcode:0x0 func:megasas_transition_to_ready > > [ 106.695186] megaraid_sas 0000:03:00.0: System Register set: > > [ 106.889787] megaraid_sas 0000:03:00.0: Failed to transition controller > to ready for scsi0. > > [ 106.910475] megaraid_sas 0000:03:00.0: Failed from megasas_init_fw > 6407 > > estuary:/$ > > > > With the series, now the kernel has direct mapping for the dev as > > below, > > > > estuary:/$ cat /sys/kernel/iommu_groups/0/reserved_regions > > 0x0000000008000000 0x00000000080fffff msi > > 0x0000000027b00000 0x00000000286fffff direct > > 0x00000000e6400000 0x00000000e64fffff direct > > estuary:/$ > > > > .... > > [ 12.254318] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is > called outbound_intr_mask:0x40000009 > > [ 12.739089] megaraid_sas 0000:03:00.0: FW provided > supportMaxExtLDs: 0 max_lds: 32 > > [ 12.746628] megaraid_sas 0000:03:00.0: controller type : > iMR(0MB) > > [ 12.752694] megaraid_sas 0000:03:00.0: Online Controller Reset(OCR) : > Enabled > > [ 12.759798] megaraid_sas 0000:03:00.0: Secure JBOD support : Yes > > [ 12.765778] megaraid_sas 0000:03:00.0: NVMe passthru support : Yes > > [ 12.771931] megaraid_sas 0000:03:00.0: FW provided TM > TaskAbort/Reset timeou: 6 secs/60 secs > > [ 12.780503] megaraid_sas 0000:03:00.0: JBOD sequence map > support : Yes > > [ 12.787000] megaraid_sas 0000:03:00.0: PCI Lane Margining > support : No > > [ 12.819179] megaraid_sas 0000:03:00.0: NVME page size : > (4096) > > [ 12.825672] megaraid_sas 0000:03:00.0: megasas_enable_intr_fusion is > called outbound_intr_mask:0x40000000 > > [ 12.835199] megaraid_sas 0000:03:00.0: INIT adapter done > > [ 12.873932] megaraid_sas 0000:03:00.0: pci id : > (0x1000)/(0x0017)/(0x19e5)/(0xd213) > > [ 12.881644] megaraid_sas 0000:03:00.0: unevenspan support : no > > [ 12.887451] megaraid_sas 0000:03:00.0: firmware crash dump : no > > [ 12.893344] megaraid_sas 0000:03:00.0: JBOD sequence map : > enabled > > > > RAID controller init is now success and can detect the drives > > attached as well. > > > > Jon Nettleton (1): > > iommu/arm-smmu: Get associated RMR info and install bypass SMR > > > > Shameer Kolothum (7): > > ACPI/IORT: Add support for RMR node parsing > > iommu/dma: Introduce generic helper to retrieve RMR info > > ACPI/IORT: Add a helper to retrieve RMR memory regions > > iommu/arm-smmu-v3: Introduce strtab init helper > > iommu/arm-smmu-v3: Add bypass flag to arm_smmu_write_strtab_ent() > > iommu/arm-smmu-v3: Get associated RMR info and install bypass STE > > iommu/dma: Reserve any RMR regions associated with a dev > > > > drivers/acpi/arm64/iort.c | 144 > +++++++++++++++++++- > > drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 69 +++++++--- > > drivers/iommu/arm/arm-smmu/arm-smmu.c | 64 +++++++++ > > drivers/iommu/dma-iommu.c | 102 +++++++++++++- > > include/linux/acpi_iort.h | 7 + > > include/linux/dma-iommu.h | 10 ++ > > include/linux/iommu.h | 19 +++ > > 7 files changed, 392 insertions(+), 23 deletions(-) > >
