> From: Jason Gunthorpe <jgg@xxxxxxxxxx> > Sent: Tuesday, April 6, 2021 8:21 PM > > On Tue, Apr 06, 2021 at 01:02:05AM +0000, Tian, Kevin wrote: > > > From: Jason Gunthorpe <jgg@xxxxxxxxxx> > > > Sent: Tuesday, April 6, 2021 7:40 AM > > > > > > On Fri, Apr 02, 2021 at 07:58:02AM +0000, Tian, Kevin wrote: > > > > > From: Jason Gunthorpe <jgg@xxxxxxxxxx> > > > > > Sent: Thursday, April 1, 2021 9:47 PM > > > > > > > > > > On Thu, Apr 01, 2021 at 01:43:36PM +0000, Liu, Yi L wrote: > > > > > > > From: Jason Gunthorpe <jgg@xxxxxxxxxx> > > > > > > > Sent: Thursday, April 1, 2021 9:16 PM > > > > > > > > > > > > > > On Thu, Apr 01, 2021 at 01:10:48PM +0000, Liu, Yi L wrote: > > > > > > > > > From: Jason Gunthorpe <jgg@xxxxxxxxxx> > > > > > > > > > Sent: Thursday, April 1, 2021 7:47 PM > > > > > > > > [...] > > > > > > > > > I'm worried Intel views the only use of PASID in a guest is with > > > > > > > > > ENQCMD, but that is not consistent with the industry. We need > to > > > see > > > > > > > > > normal nested PASID support with assigned PCI VFs. > > > > > > > > > > > > > > > > I'm not quire flow here. Intel also allows PASID usage in guest > > > without > > > > > > > > ENQCMD. e.g. Passthru a PF to guest, and use PASID on it > without > > > > > > > ENQCMD. > > > > > > > > > > > > > > Then you need all the parts, the hypervisor calls from the vIOMMU, > > > and > > > > > > > you can't really use a vPASID. > > > > > > > > > > > > This is a diagram shows the vSVA setup. > > > > > > > > > > I'm not talking only about vSVA. Generic PASID support with arbitary > > > > > mappings. > > > > > > > > > > And how do you deal with the vPASID vs pPASID issue if the system > has > > > > > a mix of physical devices and mdevs? > > > > > > > > > > > > > We plan to support two schemes. One is vPASID identity-mapped to > > > > pPASID then the mixed scenario just works, with the limitation of > > > > lacking of live migration support. The other is non-identity-mapped > > > > scheme, where live migration is supported but physical devices and > > > > mdevs should not be mixed in one VM if both expose SVA capability > > > > (requires some filtering check in Qemu). > > > > > > That just becomes "block vPASID support if any device that > > > doesn't use ENQCMD is plugged into the guest" > > > > The limitation is only for physical device. and in reality it is not that > > bad. To support live migration with physical device we anyway need > > additional work to migrate the device state (e.g. based on Max's work), > > then it's not unreasonable to also mediate guest programming of > > device specific PASID register to enable vPASID (need to translate in > > the whole VM lifespan but likely is not a hot path). > > IMHO that is pretty unreasonable.. More likely we end up with vPASID > tables in each migratable device like KVM has. just like mdev needs to maintain allowed PASID list, this extends it to all migratable devices. > > > > Which needs a special VFIO capability of some kind so qemu knows to > > > block it. This really needs to all be layed out together so someone > > > can understand it :( > > > > Or could simply based on whether the VFIO device supports live migration. > > You need to define affirmative caps that indicate that vPASID will be > supported by the VFIO device. Yes, this is required as I acked in another mail. > > > > Why doesn't the siov cookbook explaining this stuff?? > > > > > > > We hope the /dev/ioasid can support both schemes, with the minimal > > > > requirement of allowing userspace to tag a vPASID to a pPASID and > > > > allowing mdev to translate vPASID into pPASID, i.e. not assuming that > > > > the guest will always use pPASID. > > > > > > What I'm a unclear of is if /dev/ioasid even needs to care about > > > vPASID or if vPASID is just a hidden artifact of the KVM connection to > > > setup the translation table and the vIOMMU driver in qemu. > > > > Not just for KVM. Also required by mdev, which needs to translate > > vPASID into pPASID when ENQCMD is not used. > > Do we have any mdev's that will do this? definitely. Actually any mdev which doesn't do ENQCMD needs to do this. In normal case, the PASID is programmed to a MMIO register (or in-memory context) associate with the backend resource of the mdev. The value programmed from the guest is vPASID, thus must be translated into pPASID before updating the physical register. > > > should only care about the operations related to pPASID. VFIO could > > carry vPASID information to mdev. > > It depends how common this is, I suppose > based on above I think it's a common case. Thanks Kevin
