On 5/8/20 11:35 PM, Yicong Yang wrote:
Hi Jay, Kuppuswamy
On 2020/5/1 4:41, Jay Vosburgh wrote:
"Kuppuswamy, Sathyanarayanan" wrote:
Hi Jay,
On 4/29/20 6:15 PM, Kuppuswamy, Sathyanarayanan wrote:
On 4/29/20 5:42 PM, Jay Vosburgh wrote:
Commit 6d2c89441571 ("PCI/ERR: Update error status after
reset_link()"), introduced a regression, as pcie_do_recovery will
discard the status result from report_frozen_detected. This can cause a
failure to recover if _NEED_RESET is returned by report_frozen_detected
and report_slot_reset is not invoked.
Such an event can be induced for testing purposes by reducing
the Max_Payload_Size of a PCIe bridge to less than that of a device
downstream from the bridge, and then initating I/O through the device,
resulting in oversize transactions. In the presence of DPC, this
results in a containment event and attempted reset and recovery via
pcie_do_recovery. After 6d2c89441571 report_slot_reset is not invoked,
and the device does not recover.
I think this issue is related to the issue discussed in following
thread (DPC non-hotplug support).
https://lkml.org/lkml/2020/3/28/328
If my assumption is correct, you are dealing with devices which are
not hotplug capable. If the devices are hotplug capable then you don't
need to proceed to report_slot_reset(), since hotplug handler will
remove/re-enumerate the devices correctly.
Correct, this particular device (a network card) is in a
non-hotplug slot.
Can you check whether following fix works for you?
Yes, it does.
I fixed up the whitespace and made a minor change to add braces
in what look like the correct places around the "if (reset_link)" block;
the patch I tested with is below. I'll also install this on another
machine with hotplug capable slots to test there as well.
diff --git a/drivers/pci/pcie/err.c b/drivers/pci/pcie/err.c
index 14bb8f54723e..db80e1ecb2dc 100644
--- a/drivers/pci/pcie/err.c
+++ b/drivers/pci/pcie/err.c
@@ -165,13 +165,24 @@ pci_ers_result_t pcie_do_recovery(struct pci_dev *dev,
pci_dbg(dev, "broadcast error_detected message\n");
if (state == pci_channel_io_frozen) {
pci_walk_bus(bus, report_frozen_detected, &status);
- status = reset_link(dev);
- if (status != PCI_ERS_RESULT_RECOVERED) {
+ status = PCI_ERS_RESULT_NEED_RESET;
+ } else {
+ pci_walk_bus(bus, report_normal_detected, &status);
+ }
+
+ if (status == PCI_ERS_RESULT_NEED_RESET) {
+ if (reset_link) {
+ if (reset_link(dev) != PCI_ERS_RESULT_RECOVERED)
+ status = PCI_ERS_RESULT_DISCONNECT;
+ } else {
+ if (pci_bus_error_reset(dev))
+ status = PCI_ERS_RESULT_DISCONNECT;
+ }
+
The PCI_ERS_RESULT_NEED_RESET may indicate that the driver requires a *slot* reset.
With this patch, seems later slot reset broadcast may not be performed.
Slot reset wont be performed only if reset_link or pci_bus_error_reset
returns error. Otherwise, we will still call pci_slot_reset later.
if (status == PCI_ERS_RESULT_NEED_RESET) {
status = PCI_ERS_RESULT_RECOVERED;
pci_dbg(dev, "broadcast slot_reset message\n");
pci_walk_bus(bus, report_slot_reset, &status);
}
One minor question, currently the callers of pcie_do_recovery() will always pass a
reset_link pointer, so is the condition necessary?
Yes, currently we don't need it. I added it to cover future use cases.
But we can remove it if not needed.
Yicong
+ if (status == PCI_ERS_RESULT_DISCONNECT) {
pci_warn(dev, "link reset failed\n");
goto failed;
}
- } else {
- pci_walk_bus(bus, report_normal_detected, &status);
}
if (status == PCI_ERS_RESULT_CAN_RECOVER) {
pci_bus_error_reset
-J
This includes support for bus_reset in recovery function itself.
index 14bb8f54723e..c9eaab68ab7a 100644
--- a/drivers/pci/pcie/err.c
+++ b/drivers/pci/pcie/err.c
@@ -165,13 +165,23 @@ pci_ers_result_t pcie_do_recovery(struct pci_dev *dev,
pci_dbg(dev, "broadcast error_detected message\n");
if (state == pci_channel_io_frozen) {
if (state == pci_channel_io_frozen) {
pci_walk_bus(bus, report_frozen_detected, &status);
- status = reset_link(dev);
- if (status != PCI_ERS_RESULT_RECOVERED) {
+ status = PCI_ERS_RESULT_NEED_RESET;
+ } else {
+ pci_walk_bus(bus, report_normal_detected, &status);
+ }
+
+ if (status == PCI_ERS_RESULT_NEED_RESET) {
+ if (reset_link)
+ if (reset_link(dev) != PCI_ERS_RESULT_RECOVERED)
+ status = PCI_ERS_RESULT_DISCONNECT;
+ else
+ if (pci_bus_error_reset(dev))
+ status = PCI_ERS_RESULT_DISCONNECT;
+
+ if (status == PCI_ERS_RESULT_DISCONNECT) {
pci_warn(dev, "link reset failed\n");
goto failed;
}
- } else {
- pci_walk_bus(bus, report_normal_detected, &status);
}
if (status == PCI_ERS_RESULT_CAN_RECOVER) {
Inspection shows a similar path is plausible for a return of
_CAN_RECOVER and the invocation of report_mmio_enabled.
Resolve this by preserving the result of report_frozen_detected if
reset_link does not return _DISCONNECT.
Fixes: 6d2c89441571 ("PCI/ERR: Update error status after reset_link()")
Signed-off-by: Jay Vosburgh <jay.vosburgh@xxxxxxxxxxxxx>
---
drivers/pci/pcie/err.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/drivers/pci/pcie/err.c b/drivers/pci/pcie/err.c
index 14bb8f54723e..e4274562f3a0 100644
--- a/drivers/pci/pcie/err.c
+++ b/drivers/pci/pcie/err.c
@@ -164,10 +164,17 @@ pci_ers_result_t pcie_do_recovery(struct pci_dev
*dev,
pci_dbg(dev, "broadcast error_detected message\n");
if (state == pci_channel_io_frozen) {
+ pci_ers_result_t status2;
+
pci_walk_bus(bus, report_frozen_detected, &status);
- status = reset_link(dev);
- if (status != PCI_ERS_RESULT_RECOVERED) {
+ /* preserve status from report_frozen_detected to
+ * insure report_mmio_enabled or report_slot_reset are
+ * invoked even if reset_link returns _RECOVERED.
+ */
+ status2 = reset_link(dev);
+ if (status2 != PCI_ERS_RESULT_RECOVERED) {
pci_warn(dev, "link reset failed\n");
+ status = status2;
goto failed;
}
} else {
---
-Jay Vosburgh, jay.vosburgh@xxxxxxxxxxxxx
.
