> From: Baolu Lu <baolu.lu@xxxxxxxxxxxxxxx> > Sent: Thursday, May 11, 2023 7:34 PM > > On 5/11/23 3:27 PM, Tian, Kevin wrote: > >> From: Alex Williamson<alex.williamson@xxxxxxxxxx> > >> Sent: Thursday, May 11, 2023 1:25 AM > >> > >> On Tue, 9 May 2023 08:34:53 +0000 > >> "Tian, Kevin"<kevin.tian@xxxxxxxxx> wrote: > >> > >>> According to PCIe spec (7.8.9 PASID Extended Capability Structure): > >>> > >>> The PASID configuration of the single non-VF Function representing > >>> the device is also used by all VFs in the device. A PF is permitted > >>> to implement the PASID capability, but VFs must not implement it. > >>> > >>> To enable PASID on VF then one open is where to locate the PASID > >>> capability in VF's vconfig space. vfio-pci doesn't know which offset > >>> may contain VF specific config registers. Finding such offset must > >>> come from a device specific knowledge. > >> Backup for a moment, VFs are governed by the PASID capability on the > >> PF. The PASID capability exposes control registers that imply the > >> ability to manage various feature enable bits. The VF owner does not > >> have privileges to manipulate those bits. For example, the PASID Enable > >> bit should restrict the endpoint from sending TLPs with a PASID prefix, > >> but this can only be changed at the PF level for all associated VFs. > >> > >> The protocol specified in 7.8.9.3 defines this enable bit as RW. How do > >> we virtualize that? Either it's virtualized to be read-only and we > >> violate the spec or we allow it to be read-write and it has no effect, > >> which violates the spec. > >> > > Currently the PASID cap is enabled by default when a device is probed > > by iommu driver. Leaving it enabled in PF while guest wants it disabled > > in VF is harmless. W/o proper setup in iommu side the VF cannot > > do real work with PASID. > > [sorry for partial reply] > > I am attempting to move PASID enabling/disabling out of the iommu > driver and give its control to the device driver. One puzzle thing is > that PCI spec requires PASID control bits not changed once the ATS is > is enabled. So I also need to add iommu interfaces to enable/disable > ATS on devices. > > By default, the ATS is enabled by the iommu subsystem to utilize the > device TLB, the device driver needs to disable it before change the > PASID control bits and re-enable it afterwards. > ATS is also relied on by PRS. Not sure how that will be affected when ATS is dynamically turned on/off. and PRS is not tied to PASID. Jason, is it still a strong requirement to have driver-opt model for pasid enabling? iirc it's first raised in a discussion for an AMD GPU scenario [1] [1] https://lore.kernel.org/linux-pci/38a7baf4-9b3b-154b-f764-fa8cfa600858@xxxxxxxxxxxxxxx/
