On 2024/1/29 17:06, Tian, Kevin wrote:
From: Ethan Zhao <haifeng.zhao@xxxxxxxxxxxxxxx> Sent: Monday, January 29, 2024 11:49 AM Because surprise removal could happen anytime, e.g. user could request safe removal to EP(endpoint device) via sysfs and brings its link down to do surprise removal cocurrently. such aggressive cases would cause ATS invalidation request issued to non-existence target device, then deadly loop to retry that request after ITE fault triggered in interrupt context. this patch aims to optimize the ITE handling by checking the target device presence state to avoid retrying the timeout request blindly, thus avoid hard lockup or system hang. Signed-off-by: Ethan Zhao <haifeng.zhao@xxxxxxxxxxxxxxx> --- drivers/iommu/intel/dmar.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c index 814134e9aa5a..2e214b43725c 100644 --- a/drivers/iommu/intel/dmar.c +++ b/drivers/iommu/intel/dmar.c @@ -1272,6 +1272,7 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index, { u32 fault; int head, tail; + u64 iqe_err, ite_sid; struct q_inval *qi = iommu->qi; int shift = qi_shift(iommu); @@ -1316,6 +1317,13 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index, tail = readl(iommu->reg + DMAR_IQT_REG); tail = ((tail >> shift) - 1 + QI_LENGTH) % QI_LENGTH; + /* + * SID field is valid only when the ITE field is Set in FSTS_REG + * see Intel VT-d spec r4.1, section 11.4.9.9 + */ + iqe_err = dmar_readq(iommu->reg + DMAR_IQER_REG); + ite_sid = DMAR_IQER_REG_ITESID(iqe_err); + writel(DMA_FSTS_ITE, iommu->reg + DMAR_FSTS_REG); pr_info("Invalidation Time-out Error (ITE) cleared\n"); @@ -1325,6 +1333,16 @@ static int qi_check_fault(struct intel_iommu *iommu, int index, int wait_index, head = (head - 2 + QI_LENGTH) % QI_LENGTH; } while (head != tail); + /* + * If got ITE, we need to check if the sid of ITE is the same as + * current ATS invalidation target device, if yes, don't try this + * request anymore if the target device isn't present. + * 0 value of ite_sid means old VT-d device, no ite_sid value. + */ + if (pdev && ite_sid && !pci_device_is_present(pdev) && + ite_sid == pci_dev_id(pci_physfn(pdev))) + return -ETIMEDOUT; +since the hardware already reports source id leading to timeout, can't we just find the pci_dev according to reported ite_sid? this is a slow path (either due to device in bad state or removed) hence it's not necessary to add more intelligence to pass the pci_dev in, leading to only a partial fix can be backported. It's also more future-proof, say if one day the driver allows batching invalidation requests for multiple devices then no need to pass in a list of devices. Then it's easier to backport a full fix.
May consider pci_get_domain_bus_and_slot() or pci_find_bus()/pci_get_slot(). But I doubt if the pci_dev is still tracked in the bus or a kind of dev list in the device hot removal case. So Ethan may need to check. Regards, Yi Liu -- Regards, Yi Liu
