IOMMU drivers only need to flush their PRI queue when faults might be
pending. According to the PCIe spec (quoted below) this only happens
when using the "Stop Marker" method. Otherwise the function waits for
pending faults before signaling to the device driver that it can
unbind().
Add the IOMMU_UNBIND_FAULT_PENDING flags to unbind(), to tell the IOMMU
driver whether it's worth flushing the queue.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@xxxxxxxxxx>
---
include/linux/iommu.h | 31 +++++++++++++++++++++++++++++++
drivers/iommu/intel/svm.c | 3 ++-
drivers/iommu/iommu.c | 5 ++++-
3 files changed, 37 insertions(+), 2 deletions(-)
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 26c1358a2a37..fd9630b1240d 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -163,6 +163,37 @@ enum iommu_dev_features {
#define IOMMU_PASID_INVALID (-1U)
+/*
+ * Indicate that a device stops using a PASID by issuing a Stop Marker Message.
+ * From PCIe 4.0r1.0, 10.4.1.2 Managing PASID TLP Prefix Usage:
+ *
+ * "To stop without using a Stop Marker Message, the Function shall:
+ * 1. Stop queueing new Page Request Messages for this PASID.
+ * 2. Finish transmitting any multi-page Page Request Messages for this PASID
+ * (i.e. send the Page Request Message with the L bit Set).
+ * 3. Wait for PRG Response Messages associated any outstanding Page Request
+ * Messages for the PASID.
+ * 4. Indicate that the PASID has stopped using a device specific mechanism.
+ * This mechanism must indicate that a Stop Marker Message will not be
+ * generated.
+ * To stop with the use of a Stop Marker Message the Function shall:
+ * [1. and 2. are the same]
+ * 3. Internally mark all outstanding Page Request Messages for this PASID as
+ * stale. PRG Response Messages associated with these requests will return
+ * Page Request Allocation credits and PRG Index values but are otherwise
+ * ignored.
+ * 4. Indicate that the PASID has stopped using a device specific mechanism.
+ * This mechanism must indicate that a Stop Marker Message will be
+ * generated.
+ * 5. Send a Stop Marker Message to indicate to the host that all subsequent
+ * Page Request Messages for this PASID are for a new use of the PASID
+ * value."
+ *
+ * If the device indicates that the Stop Marker Message will be generated, the
+ * device driver should set the IOMMU_UNBIND_FAULT_PENDING flag.
+ */
+#define IOMMU_UNBIND_FAULT_PENDING (1UL << 0)
+
#ifdef CONFIG_IOMMU_API
/**
diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c
index 700b05612af9..aa1fcb66fa95 100644
--- a/drivers/iommu/intel/svm.c
+++ b/drivers/iommu/intel/svm.c
@@ -680,7 +680,8 @@ static int intel_svm_unbind_mm(struct device *dev, u32 pasid,
* hard to be as defensive as we might like. */
intel_pasid_tear_down_entry(iommu, dev,
svm->pasid, false);
- intel_svm_drain_prq(dev, svm->pasid);
+ if (flags & IOMMU_UNBIND_FAULT_PENDING)
+ intel_svm_drain_prq(dev, svm->pasid);
kfree_rcu(sdev, rcu);
if (list_empty(&svm->devs)) {
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 741c463095a8..eede0592a2c0 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -2995,7 +2995,10 @@ EXPORT_SYMBOL_GPL(iommu_sva_bind_device);
*
* Put reference to a bond between device and address space. The device should
* not be issuing any more transaction for this PASID. All outstanding page
- * requests for this PASID must have been flushed to the IOMMU.
+ * requests for this PASID must have been completed, or flushed to the IOMMU. If
+ * they have not been completed, for example when using a Stop Marker Message to
+ * stop PASID in a PCIe device, then the caller must set the flag
+ * %IOMMU_UNBIND_FAULT_PENDING.
*
* Returns 0 on success, or an error value
*/
--
2.28.0
