On 2/2/24 3:34 AM, Jason Gunthorpe wrote:
On Wed, Jan 31, 2024 at 02:21:20PM +0800, Baolu Lu wrote:
An rbtree for IOMMU device data for the VT-d driver would be beneficial.
It also benefits other paths of fault handling, such as the I/O page
fault handling path, where it currently still relies on the PCI
subsystem to convert a RID value into a pci_device structure.
Given that such an rbtree would be helpful for multiple individual
drivers that handle PCI devices, it seems valuable to implement it in
the core?
rbtree is already supposed to be a re-usable library.
There is already good helper support in rbtree to make things easy to
implement. I see arm hasn't used them yet, it should look something
like this:
I have posted a similar implementation for the vt-d driver here:
https://lore.kernel.org/linux-iommu/20240215072249.4465-1-baolu.lu@xxxxxxxxxxxxxxx/
Based on this implementation, only patches 1 and 2 are required. The
last patch could be like below (code compiled but not tested, comments
not changed yet):
diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c
index f9b63c2875f7..30a659a4d3ed 100644
--- a/drivers/iommu/intel/dmar.c
+++ b/drivers/iommu/intel/dmar.c
@@ -1272,6 +1272,7 @@ static int qi_check_fault(struct intel_iommu
*iommu, int index, int wait_index)
{
u32 fault;
int head, tail;
+ u64 iqe_err, ite_sid;
struct q_inval *qi = iommu->qi;
int shift = qi_shift(iommu);
@@ -1316,6 +1317,13 @@ static int qi_check_fault(struct intel_iommu
*iommu, int index, int wait_index)
tail = readl(iommu->reg + DMAR_IQT_REG);
tail = ((tail >> shift) - 1 + QI_LENGTH) % QI_LENGTH;
+ /*
+ * SID field is valid only when the ITE field is Set in
FSTS_REG
+ * see Intel VT-d spec r4.1, section 11.4.9.9
+ */
+ iqe_err = dmar_readq(iommu->reg + DMAR_IQER_REG);
+ ite_sid = DMAR_IQER_REG_ITESID(iqe_err);
+
writel(DMA_FSTS_ITE, iommu->reg + DMAR_FSTS_REG);
pr_info("Invalidation Time-out Error (ITE) cleared\n");
@@ -1325,6 +1333,19 @@ static int qi_check_fault(struct intel_iommu
*iommu, int index, int wait_index)
head = (head - 2 + QI_LENGTH) % QI_LENGTH;
} while (head != tail);
+ /*
+ * If got ITE, we need to check if the sid of ITE is the
same as
+ * current ATS invalidation target device, if yes, don't
try this
+ * request anymore if the target device isn't present.
+ * 0 value of ite_sid means old VT-d device, no ite_sid
value.
+ */
+ if (ite_sid) {
+ struct device *dev = device_rbtree_find(iommu,
ite_sid);
+
+ if (!dev || !pci_device_is_present(to_pci_dev(dev)))
+ return -ETIMEDOUT;
+ }
+
if (qi->desc_status[wait_index] == QI_ABORT)
return -EAGAIN;
}
Best regards,
baolu
