On Thu, Aug 15, 2019 at 03:39:03PM -0700, Kuppuswamy Sathyanarayanan wrote: > On 8/15/19 3:20 PM, Bjorn Helgaas wrote: > > [+cc Joerg, David, iommu list: because IOMMU drivers are the only > > callers of pci_enable_pri() and pci_enable_pasid()] > > > > On Thu, Aug 01, 2019 at 05:06:01PM -0700, sathyanarayanan.kuppuswamy@xxxxxxxxxxxxxxx wrote: > > > From: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@xxxxxxxxxxxxxxx> > > > > > > When IOMMU tries to enable Page Request Interface (PRI) for VF device > > > in iommu_enable_dev_iotlb(), it always fails because PRI support for > > > PCIe VF device is currently broken. Current implementation expects > > > the given PCIe device (PF & VF) to implement PRI capability before > > > enabling the PRI support. But this assumption is incorrect. As per PCIe > > > spec r4.0, sec 9.3.7.11, all VFs associated with PF can only use the > > > PRI of the PF and not implement it. Hence we need to create exception > > > for handling the PRI support for PCIe VF device. > > > > > > Also, since PRI is a shared resource between PF/VF, following rules > > > should apply. > > > > > > 1. Use proper locking before accessing/modifying PF resources in VF > > > PRI enable/disable call. > > > 2. Use reference count logic to track the usage of PRI resource. > > > 3. Disable PRI only if the PRI reference count (pri_ref_cnt) is zero. > > Wait, why do we need this at all? I agree the spec says VFs may not > > implement PRI or PASID capabilities and that VFs use the PRI and > > PASID of the PF. > > > > But why do we need to support pci_enable_pri() and pci_enable_pasid() > > for VFs? There's nothing interesting we can *do* in the VF, and > > passing it off to the PF adds all this locking mess. For VFs, can we > > just make them do nothing or return -EINVAL? What functionality would > > we be missing if we did that? > > Currently PRI/PASID capabilities are not enabled by default. IOMMU can > enable PRI/PASID for VF first (and not enable it for PF). In this case, > doing nothing for VF device will break the functionality. What is the path where we can enable PRI/PASID for VF but not for the PF? The call chains leading to pci_enable_pri() go through the iommu_ops.add_device interface, which makes me think this is part of the device enumeration done by the PCI core, and in that case I would think this it should be done for the PF before VFs. But maybe this path isn't exercised until a driver does a DMA map or something similar? Bjorn
