This is a note to let you know that I've just added the patch titled
PCI: vmd: Disable bridge window for domain reset
to the 6.5-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary
The filename of the patch is:
pci-vmd-disable-bridge-window-for-domain-reset.patch
and it can be found in the queue-6.5 subdirectory.
If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@xxxxxxxxxxxxxxx> know about it.
commit 337cbbca1309cf319efcb19aa03af7589e2b0506
Author: Nirmal Patel <nirmal.patel@xxxxxxxxxxxxxxx>
Date: Thu Aug 10 17:50:29 2023 -0400
PCI: vmd: Disable bridge window for domain reset
[ Upstream commit f73eedc90bf73d48e8368e6b0b4ad76a7fffaef7 ]
During domain reset process vmd_domain_reset() clears PCI
configuration space of VMD root ports. But certain platform
has observed following errors and failed to boot.
...
DMAR: VT-d detected Invalidation Queue Error: Reason f
DMAR: VT-d detected Invalidation Time-out Error: SID ffff
DMAR: VT-d detected Invalidation Completion Error: SID ffff
DMAR: QI HEAD: UNKNOWN qw0 = 0x0, qw1 = 0x0
DMAR: QI PRIOR: UNKNOWN qw0 = 0x0, qw1 = 0x0
DMAR: Invalidation Time-out Error (ITE) cleared
The root cause is that memset_io() clears prefetchable memory base/limit
registers and prefetchable base/limit 32 bits registers sequentially.
This seems to be enabling prefetchable memory if the device disabled
prefetchable memory originally.
Here is an example (before memset_io()):
PCI configuration space for 10000:00:00.0:
86 80 30 20 06 00 10 00 04 00 04 06 00 00 01 00
00 00 00 00 00 00 00 00 00 01 01 00 00 00 00 20
00 00 00 00 01 00 01 00 ff ff ff ff 75 05 00 00
...
So, prefetchable memory is ffffffff00000000-575000fffff, which is
disabled. When memset_io() clears prefetchable base 32 bits register,
the prefetchable memory becomes 0000000000000000-575000fffff, which is
enabled and incorrect.
Here is the quote from section 7.5.1.3.9 of PCI Express Base 6.0 spec:
The Prefetchable Memory Limit register must be programmed to a smaller
value than the Prefetchable Memory Base register if there is no
prefetchable memory on the secondary side of the bridge.
This is believed to be the reason for the failure and in addition the
sequence of operation in vmd_domain_reset() is not following the PCIe
specs.
Disable the bridge window by executing a sequence of operations
borrowed from pci_disable_bridge_window() and pci_setup_bridge_io(),
that comply with the PCI specifications.
Link: https://lore.kernel.org/r/20230810215029.1177379-1-nirmal.patel@xxxxxxxxxxxxxxx
Signed-off-by: Nirmal Patel <nirmal.patel@xxxxxxxxxxxxxxx>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@xxxxxxxxxx>
Signed-off-by: Sasha Levin <sashal@xxxxxxxxxx>
diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index e718a816d4814..ad56df98b8e63 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -541,8 +541,23 @@ static void vmd_domain_reset(struct vmd_dev *vmd)
PCI_CLASS_BRIDGE_PCI))
continue;
- memset_io(base + PCI_IO_BASE, 0,
- PCI_ROM_ADDRESS1 - PCI_IO_BASE);
+ /*
+ * Temporarily disable the I/O range before updating
+ * PCI_IO_BASE.
+ */
+ writel(0x0000ffff, base + PCI_IO_BASE_UPPER16);
+ /* Update lower 16 bits of I/O base/limit */
+ writew(0x00f0, base + PCI_IO_BASE);
+ /* Update upper 16 bits of I/O base/limit */
+ writel(0, base + PCI_IO_BASE_UPPER16);
+
+ /* MMIO Base/Limit */
+ writel(0x0000fff0, base + PCI_MEMORY_BASE);
+
+ /* Prefetchable MMIO Base/Limit */
+ writel(0, base + PCI_PREF_LIMIT_UPPER32);
+ writel(0x0000fff0, base + PCI_PREF_MEMORY_BASE);
+ writel(0xffffffff, base + PCI_PREF_BASE_UPPER32);
}
}
}
