On 04/22/2018 05:48 AM, Borislav Petkov wrote:
On Thu, Apr 19, 2018 at 05:55:08PM -0500, Alex G. wrote:
How does such an error look like, in detail?
It's green on the soft side, with lots of red accents, as well as some
textured white shades:
[ 51.414616] pciehp 0000:b0:06.0:pcie204: Slot(176): Link Down
[ 51.414634] pciehp 0000:b0:05.0:pcie204: Slot(179): Link Down
[ 52.703343] FIRMWARE BUG: Firmware sent fatal error that we were able
to correct
[ 52.703345] BROKEN FIRMWARE: Complain to your hardware vendor
[ 52.703347] {1}[Hardware Error]: Hardware error from APEI Generic
Hardware Error Source: 1
[ 52.703358] pciehp 0000:b0:06.0:pcie204: Slot(176): Link Up
[ 52.711616] {1}[Hardware Error]: event severity: fatal
[ 52.716754] {1}[Hardware Error]: Error 0, type: fatal
[ 52.721891] {1}[Hardware Error]: section_type: PCIe error
[ 52.727463] {1}[Hardware Error]: port_type: 6, downstream switch port
[ 52.734075] {1}[Hardware Error]: version: 3.0
[ 52.738607] {1}[Hardware Error]: command: 0x0407, status: 0x0010
[ 52.744786] {1}[Hardware Error]: device_id: 0000:b0:06.0
[ 52.750271] {1}[Hardware Error]: slot: 4
[ 52.754371] {1}[Hardware Error]: secondary_bus: 0xb3
[ 52.759509] {1}[Hardware Error]: vendor_id: 0x10b5, device_id: 0x9733
[ 52.766123] {1}[Hardware Error]: class_code: 000406
[ 52.771182] {1}[Hardware Error]: bridge: secondary_status: 0x0000,
control: 0x0003
[ 52.779038] pcieport 0000:b0:06.0: aer_status: 0x00100000, aer_mask:
0x01a10000
[ 52.782303] nvme0n1: detected capacity change from 3200631791616 to 0
[ 52.786348] pcieport 0000:b0:06.0: [20] Unsupported Request
[ 52.786349] pcieport 0000:b0:06.0: aer_layer=Transaction Layer,
aer_agent=Requester ID
[ 52.786350] pcieport 0000:b0:06.0: aer_uncor_severity: 0x004eb030
[ 52.786352] pcieport 0000:b0:06.0: TLP Header: 40000001 0000020f
e12023bc 01000000
[ 52.786357] pcieport 0000:b0:06.0: broadcast error_detected message
[ 52.883895] pci 0000:b3:00.0: device has no driver
[ 52.883976] pciehp 0000:b0:06.0:pcie204: Slot(176): Link Down
[ 52.884184] pciehp 0000:b0:06.0:pcie204: Slot(176): Link Down event
queued; currently getting powered on
[ 52.967175] pciehp 0000:b0:06.0:pcie204: Slot(176): Link Up
Btw, from another discussion we're having with Yazen:
@Yazen, do you see how this error record is worth shit?
class_code: 000406
command: 0x0407, status: 0x0010
bridge: secondary_status: 0x0000, control: 0x0003
aer_status: 0x00100000, aer_mask: 0x01a10000
aer_uncor_severity: 0x004eb030
That tells you what FFS said about the error. Keep in mind that FFS has
cleared the hardware error bits, which the AER handler would normally
read from the PCI device.
those above are only some of the fields which are purely useless
undecoded. Makes me wonder what's worse for the user: dump the
half-decoded error or not dump an error at all...
It's immediately obvious if there's a glaring FFS bug and if we get
bogus data. If you distrust firmware as much as I do, then you will find
great value in having such info in the logs. It's probably not too
useful to a casual user, but then neither is a majority of the system log.
Anyway, Alex, I see this in the logs:
[ 66.581121] pciehp 0000:b0:06.0:pcie204: Slot(176): Link Down
[ 66.591939] pciehp 0000:b0:05.0:pcie204: Slot(179): Card not present
[ 66.592102] pciehp 0000:b0:06.0:pcie204: Slot(176): Card not present
and that comes from that pciehp_isr() interrupt handler AFAICT.
So there *is* a way to know that the card is not present anymore. So,
theoretically, and ignoring the code layering for now, we can connect
that error to the card not present event and then ignore the error...
You're missing the timing and assuming you will get the hotplug
interrupt. In this example, you have 22ms between the link down and
presence detect state change. This is a fairly fast removal.
Hotplug dependencies aside (you can have the kernel run without PCIe
hotplug support), I don't think you want to just linger in NMI for
dozens of milliseconds waiting for presence detect confirmation.
For enterprise SFF NVMe drives, the data lanes will disconnect before
the presence detect. FFS relies on presence detect, and these are two of
the reasons why slow removal is such a problem. You might not get a
presence detect interrupt at all.
Presence detect is optional for PCIe. PD is such a reliable heuristic,
that it guarantees worse error handling than the crackmonkey firmware. I
don't see how might be useful in a way which gives us better handling
than firmware.
Hmmm.
Hmmm
Anyway, heuristics about PCIe error recovery belong in the recovery
handler. I don't think it's smart to apply policy before we get there
Alex
--
To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
