On 2023/3/17 8:06, Tian, Kevin wrote:
From: Baolu Lu <baolu.lu@xxxxxxxxxxxxxxx>
Sent: Thursday, March 16, 2023 4:17 PM
On 2023/3/16 15:17, Tian, Kevin wrote:
From: Lu Baolu <baolu.lu@xxxxxxxxxxxxxxx>
Sent: Thursday, March 9, 2023 10:57 AM
@@ -4689,17 +4704,21 @@ static int intel_iommu_disable_iopf(struct
device
*dev)
{
struct device_domain_info *info = dev_iommu_priv_get(dev);
struct intel_iommu *iommu = info->iommu;
- int ret;
- ret = iommu_unregister_device_fault_handler(dev);
- if (ret)
- return ret;
+ if (!info->pri_enabled)
+ return -EINVAL;
- ret = iopf_queue_remove_device(iommu->iopf_queue, dev);
- if (ret)
- iommu_register_device_fault_handler(dev,
iommu_queue_iopf, dev);
+ pci_disable_pri(to_pci_dev(dev));
+ info->pri_enabled = 0;
- return ret;
+ /*
+ * With pri_enabled checked, unregistering fault handler and
+ * removing device from iopf queue should never fail.
+ */
+ iommu_unregister_device_fault_handler(dev);
+ iopf_queue_remove_device(iommu->iopf_queue, dev);
+
+ return 0;
}
PCIe spec says that clearing the enable bit doesn't mean in-fly
page requests are completed:
--
Enable (E) - This field, when set, indicates that the Page Request
Interface is allowed to make page requests. If this field is Clear,
the Page Request Interface is not allowed to issue page requests.
If both this field and the Stopped field are Clear, then the Page
Request Interface will not issue new page requests, but has
outstanding page requests that have been transmitted or are
queued for transmission
Yes. So the iommu driver should drain the in-fly PRQs.
The Intel VT-d implementation drains the PRQs when any PASID is unbound
from the iommu domain (see intel_svm_drain_prq()) before reuse. Before
disabling iopf, the device driver should unbind pasid and disable sva,
so when it comes here, the PRQ should have been drained.
Perhaps I can add below comments to make this clear:
/*
* PCIe spec states that by clearing PRI enable bit, the Page
* Request Interface will not issue new page requests, but has
* outstanding page requests that have been transmitted or are
* queued for transmission. This is supposed to be called after
* the device driver has stopped DMA, all PASIDs have been
* unbound and the outstanding PRQs have been drained.
*/
this is fine. But it should be a separate patch which removes
check of return value. It's not caused by this PRI handling move
patch.
Okay, that will be clearer.
Best regards,
baolu
