According to PCIe specification 4.0 sections 6.2.3.2.4 and 6.2.4.3, certain uncorrectable errors will signal ERR_COR instead of ERR_NONFATAL, logged as Advisory Non-Fatal Error, and set bits in both Correctable Error Status register and Uncorrectable Error Status register. Currently, when handling AER event the kernel will only look at CE status or UE status, but never both. In the Advisory Non-Fatal Error case, bits set in UE status register will not be reported and cleared until the next Fatal/Non-Fatal error arrives. For instance, before this patch series, once kernel receives an ANFE with Poisoned TLP in OS native AER mode, only status of CE will be reported and cleared: [ 148.459816] pcieport 0000:b7:02.0: AER: Corrected error received: 0000:b7:02.0 [ 148.459858] pcieport 0000:b7:02.0: PCIe Bus Error: severity=Corrected, type=Transaction Layer, (Receiver ID) [ 148.459863] pcieport 0000:b7:02.0: device [8086:0db0] error status/mask=00002000/00000000 [ 148.459868] pcieport 0000:b7:02.0: [13] NonFatalErr If the kernel receives a Malformed TLP after that, two UE will be reported, which is unexpected. Malformed TLP Header was lost since the previous ANF gated the TLP header logs: [ 170.540192] pcieport 0000:b7:02.0: PCIe Bus Error: severity=Uncorrected (Fatal), type=Transaction Layer, (Receiver ID) [ 170.552420] pcieport 0000:b7:02.0: device [8086:0db0] error status/mask=00041000/00180020 [ 170.561904] pcieport 0000:b7:02.0: [12] TLP (First) [ 170.569656] pcieport 0000:b7:02.0: [18] MalfTLP To handle this case properly, this patch set adds additional fields in aer_err_info to track both CE and UE status/mask and UE severity. This information will later be used to determine the register bits that need to be cleared. Additionally, adds more data to aer_event tracepoint, which would help to better understand ANFE and other errors for external observation. In the previous scenario, after this patch series, both CE status and related UE status will be reported and cleared after ANFE: [ 148.459816] pcieport 0000:b7:02.0: AER: Corrected error received: 0000:b7:02.0 [ 148.459858] pcieport 0000:b7:02.0: PCIe Bus Error: severity=Corrected, type=Transaction Layer, (Receiver ID) [ 148.459863] pcieport 0000:b7:02.0: device [8086:0db0] error status/mask=00002000/00000000 [ 148.459868] pcieport 0000:b7:02.0: [13] NonFatalErr [ 148.459868] pcieport 0000:b7:02.0: Uncorrectable errors that may cause Advisory Non-Fatal: [ 148.459868] pcieport 0000:b7:02.0: [18] TLP Wang, Qingshun (4): pci/aer: Store more information in aer_err_info pci/aer: Handle Advisory Non-Fatal properly pci/aer: Fetch information for FTrace ras: Trace more information in aer_event drivers/acpi/apei/ghes.c | 16 ++- drivers/cxl/core/pci.c | 15 ++- drivers/pci/pci.h | 12 ++- drivers/pci/pcie/aer.c | 188 +++++++++++++++++++++++++++------- include/linux/aer.h | 6 +- include/linux/pci.h | 27 +++++ include/ras/ras_event.h | 48 ++++++++- include/uapi/linux/pci_regs.h | 1 + 8 files changed, 266 insertions(+), 47 deletions(-) base-commit: 610a9b8f49fbcf1100716370d3b5f6f884a2835a -- 2.42.0
