On Thu, Dec 19, 2024 at 09:54:55AM -0800, Tim Harvey wrote: > Greetings, > > I have a board with an NXP IMX8MP SoC Gen3 PCI host controller > connected to a Diodes Inc PI7C9X3G606GP (imx8mp-venice-gw82xx-2x.dts) > which hangs during pci enumeration if PCIEAER is enabled. How to reproduce it? Just enable CONFIG_PCIEAER? Frank > > I've tracked this down to the enabling of fatal error reporting > (PCI_EXP_DEVCTL_FERE) on the upstream port of the PI7C9X3G606GP. In > other words if I mask that bit out of the > pcie_capability_set_word(dev, PCI_EXP_DEVCTL, PCI_EXP_AER_FLAGS) call > for that device (or disable PCI AER via CONFIG_PCIEAER or pci=noaer) > all is well. > > Here is what lspci shows for the root complex and the switch upstream port: > # lspci -n > 00:00.0 0604: 16c3:abcd (rev 01) > 01:00.0 0604: 12d8:c008 (rev 07) > 02:01.0 0604: 12d8:c008 (rev 06) > 02:02.0 0604: 12d8:c008 (rev 06) > 02:03.0 0604: 12d8:c008 (rev 06) > 02:04.0 0604: 12d8:c008 (rev 06) > 02:05.0 0604: 12d8:c008 (rev 06) > 02:06.0 0604: 12d8:c008 (rev 06) > 02:07.0 0604: 12d8:c008 (rev 06) > 09:00.0 0200: 1055:7430 (rev 11) > # lspci -s 00:00.0 -vvv > 00:00.0 PCI bridge: Synopsys, Inc. DWC_usb3 / PCIe bridge (rev 01) > (prog-if 00 [Normal decode]) > Device tree node: > /sys/firmware/devicetree/base/soc@0/pcie@33800000/pcie@0,0 > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- > ParErr- Stepping- SERR+ FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- > <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0 > Interrupt: pin A routed to IRQ 219 > Region 0: Memory at 18000000 (32-bit, non-prefetchable) [size=1M] > Bus: primary=00, secondary=01, subordinate=ff, sec-latency=0 > I/O behind bridge: f000-0fff [disabled] [16-bit] > Memory behind bridge: 18100000-182fffff [size=2M] [32-bit] > Prefetchable memory behind bridge: fff00000-000fffff [disabled] [32-bit] > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- > <TAbort- <MAbort- <SERR- <PERR- > Expansion ROM at 18300000 [virtual] [disabled] [size=64K] > BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B- > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI- D1+ D2- AuxCurrent=375mA > PME(D0+,D1+,D2-,D3hot+,D3cold+) > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [50] MSI: Enable+ Count=1/1 Maskable+ 64bit+ > Address: 0000000040101000 Data: 0000 > Masking: 00000000 Pending: 00000000 > Capabilities: [70] Express (v2) Root Port (Slot-), IntMsgNum 0 > DevCap: MaxPayload 128 bytes, PhantFunc 0 > ExtTag- RBE+ TEE-IO- > DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+ > RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- > MaxPayload 128 bytes, MaxReadReq 512 bytes > DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- > AuxPwr+ TransPend- > LnkCap: Port #0, Speed 8GT/s, Width x1, ASPM L0s L1, > Exit Latency L0s <1us, L1 unlimited > ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+ > LnkCtl: ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+ > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > LnkSta: Speed 8GT/s, Width x1 > TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt+ > RootCap: CRSVisible+ > RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- > PMEIntEna+ CRSVisible+ > RootSta: PME ReqID 0000, PMEStatus- PMEPending- > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ > NROPrPrP+ LTR- > 10BitTagComp- 10BitTagReq- OBFF Not > Supported, ExtFmt- EETLPPrefix- > EmergencyPowerReduction Not Supported, > EmergencyPowerReductionInit- > FRS- LN System CLS Not Supported, TPHComp- > ExtTPHComp- ARIFwd- > AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS- > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- ARIFwd- > AtomicOpsCtl: ReqEn- EgressBlck- > IDOReq- IDOCompl- LTR- EmergencyPowerReductionReq- > 10BitTagReq- OBFF Disabled, EETLPPrefixBlk- > LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- > Retimer- 2Retimers- DRS- > LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis- > Transmit Margin: Normal Operating Range, > EnterModifiedCompliance- ComplianceSOS- > Compliance Preset/De-emphasis: -6dB > de-emphasis, 0dB preshoot > LnkSta2: Current De-emphasis Level: -6dB, > EqualizationComplete+ EqualizationPhase1+ > EqualizationPhase2+ EqualizationPhase3+ > LinkEqualizationRequest- > Retimer- 2Retimers- CrosslinkRes: unsupported > Capabilities: [100 v2] Advanced Error Reporting > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- > UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSViol- UncorrIntErr- > BlockedTLP- AtomicOpBlocked- TLPBlockedErr- > PoisonTLPBlocked- DMWrReqBlocked- IDECheck- > MisIDETLP- PCRC_CHECK- TLPXlatBlocked- > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- > UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSViol- UncorrIntErr+ > BlockedTLP- AtomicOpBlocked- TLPBlockedErr- > PoisonTLPBlocked- DMWrReqBlocked- IDECheck- > MisIDETLP- PCRC_CHECK- TLPXlatBlocked- > UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- > UnxCmplt- RxOF+ MalfTLP+ > ECRC- UnsupReq- ACSViol- UncorrIntErr+ > BlockedTLP- AtomicOpBlocked- TLPBlockedErr- > PoisonTLPBlocked- DMWrReqBlocked- IDECheck- > MisIDETLP- PCRC_CHECK- TLPXlatBlocked- > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- > AdvNonFatalErr- CorrIntErr- HeaderOF- > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- > AdvNonFatalErr+ CorrIntErr+ HeaderOF+ > AERCap: First Error Pointer: 00, ECRCGenCap+ > ECRCGenEn- ECRCChkCap+ ECRCChkEn- > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > HeaderLog: 00000000 00000000 00000000 00000000 > RootCmd: CERptEn+ NFERptEn+ FERptEn+ > RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd- > FirstFatal- NonFatalMsg- FatalMsg- IntMsgNum 0 > ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000 > Capabilities: [148 v1] Secondary PCI Express > LnkCtl3: LnkEquIntrruptEn- PerformEqu- > LaneErrStat: 0 > Capabilities: [158 v1] L1 PM Substates > L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2- > ASPM_L1.1+ L1_PM_Substates+ > PortCommonModeRestoreTime=10us PortTPowerOnTime=10us > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- > T_CommonMode=10us > L1SubCtl2: T_PwrOn=10us > Kernel driver in use: pcieport > # lspci -s 01:00.0 -vvv > 01:00.0 PCI bridge: Pericom Semiconductor Device c008 (rev 07) > (prog-if 00 [Normal decode]) > Subsystem: Pericom Semiconductor Device c008 > Device tree node: > /sys/firmware/devicetree/base/soc@0/pcie@33800000/pcie@0,0/pcie@0,0 > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- > ParErr- Stepping- SERR- FastB2B- DisINTx- > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- > <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0 > Region 0: Memory at 18200000 (32-bit, non-prefetchable) [size=512K] > Bus: primary=01, secondary=02, subordinate=09, sec-latency=0 > I/O behind bridge: 0000f000-00000fff [disabled] [32-bit] > Memory behind bridge: 18100000-181fffff [size=1M] [32-bit] > Prefetchable memory behind bridge: > 00000000fff00000-00000000000fffff [disabled] [64-bit] > Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- > <TAbort- <MAbort- <SERR- <PERR- > BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B- > PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA > PME(D0+,D1-,D2-,D3hot+,D3cold+) > Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [48] MSI: Enable- Count=1/8 Maskable+ 64bit+ > Address: 0000000000000000 Data: 0000 > Masking: 00000000 Pending: 00000000 > Capabilities: [68] Express (v2) Upstream Port, IntMsgNum 0 > DevCap: MaxPayload 512 bytes, PhantFunc 0 > ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ > SlotPowerLimit 4W TEE-IO- > DevCtl: CorrErr+ NonFatalErr+ FatalErr- UnsupReq+ > RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- > MaxPayload 128 bytes, MaxReadReq 128 bytes > DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+ > AuxPwr- TransPend- > LnkCap: Port #0, Speed 8GT/s, Width x1, ASPM L1, Exit > Latency L1 <1us > ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ > LnkCtl: ASPM Disabled; LnkDisable- CommClk+ > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > LnkSta: Speed 8GT/s, Width x1 > TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- > DevCap2: Completion Timeout: Not Supported, > TimeoutDis- NROPrPrP- LTR- > 10BitTagComp- 10BitTagReq- OBFF Not > Supported, ExtFmt- EETLPPrefix- > EmergencyPowerReduction Not Supported, > EmergencyPowerReductionInit- > FRS- > AtomicOpsCap: Routing+ 32bit- 64bit- 128bitCAS- > DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- > AtomicOpsCtl: EgressBlck- > IDOReq- IDOCompl- LTR- EmergencyPowerReductionReq- > 10BitTagReq- OBFF Disabled, EETLPPrefixBlk- > LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- > Retimer- 2Retimers- DRS- > LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis- > Transmit Margin: Normal Operating Range, > EnterModifiedCompliance- ComplianceSOS- > Compliance Preset/De-emphasis: -6dB > de-emphasis, 0dB preshoot > LnkSta2: Current De-emphasis Level: -6dB, > EqualizationComplete+ EqualizationPhase1+ > EqualizationPhase2+ EqualizationPhase3+ > LinkEqualizationRequest- > Retimer- 2Retimers- CrosslinkRes: unsupported > Capabilities: [a4] Subsystem: Pericom Semiconductor Device c008 > Capabilities: [b0] MSI-X: Enable- Count=6 Masked- > Vector table: BAR=0 offset=0007f000 > PBA: BAR=0 offset=0007f080 > Capabilities: [100 v1] Advanced Error Reporting > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- > UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSViol- UncorrIntErr- > BlockedTLP- AtomicOpBlocked- TLPBlockedErr- > PoisonTLPBlocked- DMWrReqBlocked- IDECheck- > MisIDETLP- PCRC_CHECK- TLPXlatBlocked- > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- > UnxCmplt- RxOF- MalfTLP- > ECRC- UnsupReq- ACSViol- UncorrIntErr+ > BlockedTLP- AtomicOpBlocked- TLPBlockedErr- > PoisonTLPBlocked- DMWrReqBlocked- IDECheck- > MisIDETLP- PCRC_CHECK- TLPXlatBlocked- > UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- > UnxCmplt- RxOF+ MalfTLP+ > ECRC- UnsupReq- ACSViol- UncorrIntErr+ > BlockedTLP- AtomicOpBlocked- TLPBlockedErr- > PoisonTLPBlocked- DMWrReqBlocked- IDECheck- > MisIDETLP- PCRC_CHECK- TLPXlatBlocked- > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- > AdvNonFatalErr+ CorrIntErr- HeaderOF- > CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- > AdvNonFatalErr+ CorrIntErr+ HeaderOF- > AERCap: First Error Pointer: 00, ECRCGenCap+ > ECRCGenEn- ECRCChkCap+ ECRCChkEn- > MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- > HeaderLog: 00000000 00000000 00000000 00000000 > Capabilities: [130 v1] Virtual Channel > Caps: LPEVC=0 RefClk=100ns PATEntryBits=4 > Arb: Fixed- WRR32- WRR64- WRR128- > Ctrl: ArbSelect=Fixed > Status: InProgress- > VC0: Caps: PATOffset=05 MaxTimeSlots=64 RejSnoopTrans- > Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256- > Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff > Status: NegoPending- InProgress- > Port Arbitration Table <?> > Capabilities: [1a0 v1] Device Serial Number 08-16-48-96-00-00-12-d8 > Capabilities: [1b0 v1] Power Budgeting <?> > Capabilities: [1d0 v1] Multicast > McastCap: MaxGroups 64, ECRCRegen- > McastCtl: NumGroups 1, Enable- > McastBAR: IndexPos 0, BaseAddr 0000000000000000 > McastReceiveVec: 0000000000000000 > McastBlockAllVec: 0000000000000000 > McastBlockUntransVec: 0000000000000000 > McastOverlayBAR: OverlaySize 0 (disabled), BaseAddr > 0000000000000000 > Capabilities: [210 v1] Secondary PCI Express > LnkCtl3: LnkEquIntrruptEn- PerformEqu- > LaneErrStat: 0 > Capabilities: [2b0 v1] L1 PM Substates > L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- > ASPM_L1.1- L1_PM_Substates+ > L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- > L1SubCtl2: > Capabilities: [300 v1] Vendor Specific Information: ID=0000 > Rev=0 Len=560 <?> > Kernel driver in use: pcieport > > Is there anything here or any ideas on what could be the issue here? > > Best Regards, > > Tim
