On 09/18/2018 12:51 PM, Wolfram Sang wrote: > On Tue, Sep 18, 2018 at 11:15:55AM +0100, Lorenzo Pieralisi wrote: >> On Tue, May 22, 2018 at 12:05:14AM +0200, Marek Vasut wrote: >>> From: Phil Edworthy <phil.edworthy@xxxxxxxxxxx> >>> >>> The PCIe DMA controller on RCar Gen2 and earlier is on 32bit bus, >>> so limit the DMA range to 32bit. >>> >>> Signed-off-by: Phil Edworthy <phil.edworthy@xxxxxxxxxxx> >>> Signed-off-by: Marek Vasut <marek.vasut+renesas@xxxxxxxxx> >>> Cc: Arnd Bergmann <arnd@xxxxxxxx> >>> Cc: Geert Uytterhoeven <geert+renesas@xxxxxxxxx> >>> Cc: Phil Edworthy <phil.edworthy@xxxxxxxxxxx> >>> Cc: Simon Horman <horms+renesas@xxxxxxxxxxxx> >>> Cc: Wolfram Sang <wsa@xxxxxxxxxxxxx> >>> Cc: linux-renesas-soc@xxxxxxxxxxxxxxx >>> To: linux-pci@xxxxxxxxxxxxxxx >>> --- >>> NOTE: I'm aware of https://patchwork.kernel.org/patch/9495895/ , but the >>> discussion seems to have gone way off, so I'm sending this as a >>> RFC. Any feedback on how to do this limiting properly would be nice. >>> --- >>> drivers/pci/host/pcie-rcar.c | 28 ++++++++++++++++++++++++++++ >>> 1 file changed, 28 insertions(+) >> >> The issue solved by this patch was solved in a more generic way through >> this series: >> >> https://lists.linuxfoundation.org/pipermail/iommu/2018-July/028792.html >> >> I will therefore drop this patch from the PCI patch queue. > > Cool. Thanks for this series and thanks for the heads up! > > Marek, can you confirm our issue is fixed (if you haven't done already)? I assembled a setup with NVMe controller, which should be triggering this issue according to [1], but I'm not having much success getting it to work with or without this patchset. Logs are below, nvme smart-log seems to produce completely bogus results, although the device is at least recognized. Maybe there's something else that's broken ? [1] https://patchwork.kernel.org/patch/9495895/ # dmesg | egrep '(pci|nvm)' rcar-pcie fe000000.pcie: host bridge /soc/pcie@fe000000 ranges: rcar-pcie fe000000.pcie: Parsing ranges property... rcar-pcie fe000000.pcie: IO 0xfe100000..0xfe1fffff -> 0x00000000 rcar-pcie fe000000.pcie: MEM 0xfe200000..0xfe3fffff -> 0xfe200000 rcar-pcie fe000000.pcie: MEM 0x30000000..0x37ffffff -> 0x30000000 rcar-pcie fe000000.pcie: MEM 0x38000000..0x3fffffff -> 0x38000000 rcar-pcie fe000000.pcie: PCIe x1: link up rcar-pcie fe000000.pcie: Current link speed is 5 GT/s rcar-pcie fe000000.pcie: PCI host bridge to bus 0000:00 pci_bus 0000:00: root bus resource [bus 00-ff] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] pci_bus 0000:00: root bus resource [mem 0xfe200000-0xfe3fffff] pci_bus 0000:00: root bus resource [mem 0x30000000-0x37ffffff] pci_bus 0000:00: root bus resource [mem 0x38000000-0x3fffffff pref] pci_bus 0000:00: scanning bus pci 0000:00:00.0: [1912:0025] type 01 class 0x060400 pci 0000:00:00.0: enabling Extended Tags pci 0000:00:00.0: PME# supported from D0 D3hot D3cold pci 0000:00:00.0: PME# disabled pci_bus 0000:00: fixups for bus pci 0000:00:00.0: scanning [bus 01-01] behind bridge, pass 0 pci 0000:00:00.0: scanning [bus 00-00] behind bridge, pass 1 pci_bus 0000:01: scanning bus pci 0000:01:00.0: [8086:f1a5] type 00 class 0x010802 pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00003fff 64bit] pci 0000:01:00.0: 4.000 Gb/s available PCIe bandwidth, limited by 5 GT/s x1 link at 0000:00:00.0 (capable of 31.504 Gb/s with 8 GT/s x4 link) pci_bus 0000:01: fixups for bus pci_bus 0000:01: bus scan returning with max=01 pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01 pci_bus 0000:00: bus scan returning with max=01 pci 0000:00:00.0: BAR 14: assigned [mem 0xfe200000-0xfe2fffff] pci 0000:01:00.0: BAR 0: assigned [mem 0xfe200000-0xfe203fff 64bit] pci 0000:00:00.0: PCI bridge to [bus 01] pci 0000:00:00.0: bridge window [mem 0xfe200000-0xfe2fffff] pcieport 0000:00:00.0: assign IRQ: got 0 pcieport 0000:00:00.0: enabling device (0000 -> 0002) pcieport 0000:00:00.0: enabling bus mastering pcieport 0000:00:00.0: Signaling PME with IRQ 198 rcar-pcie ee800000.pcie: host bridge /soc/pcie@ee800000 ranges: rcar-pcie ee800000.pcie: Parsing ranges property... rcar-pcie ee800000.pcie: IO 0xee900000..0xee9fffff -> 0x00000000 rcar-pcie ee800000.pcie: MEM 0xeea00000..0xeebfffff -> 0xeea00000 rcar-pcie ee800000.pcie: MEM 0xc0000000..0xc7ffffff -> 0xc0000000 rcar-pcie ee800000.pcie: MEM 0xc8000000..0xcfffffff -> 0xc8000000 rcar-pcie ee800000.pcie: PCIe link down nvme 0000:01:00.0: assign IRQ: got 171 nvme nvme0: pci function 0000:01:00.0 nvme 0000:01:00.0: enabling device (0000 -> 0002) nvme 0000:01:00.0: enabling bus mastering nvme nvme0: Shutdown timeout set to 60 seconds # lspci -nvvvs 01:00.0 01:00.0 0108: 8086:f1a5 (rev 03) (prog-if 02 [NVM Express]) Subsystem: 8086:390a Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 171 Region 0: Memory at fe200000 (64-bit, non-prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [70] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+ RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- FLReset- MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend- LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L0s <1us, L1 <8us ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR+, OBFF Via message DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [b0] MSI-X: Enable- Count=16 Masked- Vector table: BAR=0 offset=00002000 PBA: BAR=0 offset=00002100 Capabilities: [100 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 14, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [158 v1] #19 Capabilities: [178 v1] Latency Tolerance Reporting Max snoop latency: 0ns Max no snoop latency: 0ns Capabilities: [180 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+ PortCommonModeRestoreTime=10us PortTPowerOnTime=10us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- T_CommonMode=0us LTR1.2_Threshold=0ns L1SubCtl2: T_PwrOn=10us Kernel driver in use: nvme lspci: Unable to load libkmod resources: error -12 # nvme smart-log /dev/nvme0 Smart Log for NVME device:nvme0 namespace-id:ffffffff critical_warning : 0 temperature : -273 C available_spare : 0% available_spare_threshold : 0% percentage_used : 0% data_units_read : 0 data_units_written : 0 host_read_commands : 0 host_write_commands : 0 controller_busy_time : 0 power_cycles : 0 power_on_hours : 0 unsafe_shutdowns : 0 media_errors : 0 num_err_log_entries : 0 Warning Temperature Time : 0 Critical Composite Temperature Time : 0 Thermal Management T1 Trans Count : 0 Thermal Management T2 Trans Count : 0 Thermal Management T1 Total Time : 0 Thermal Management T2 Total Time : 0 -- Best regards, Marek Vasut
