On Fri, Sep 27, 2024 at 3:58 PM Jason Gunthorpe <jgg@xxxxxxxxxx> wrote: > > On Fri, Sep 27, 2024 at 02:22:20PM +0000, Mostafa Saleh wrote: > > > > We don't support multi SID through this interface, at least in this > > > version. > > > > > > To do multi-sid you have to inform the VM of all the different pSIDs > > > the device has and then setup the vSID/pSID translation to map them > > > all to the HW invalidation logic. > > > > Why would the VM need to know the pSID? > > It doesn't need to know the pSID exactly, but it needs to know all the > pSIDs that exist and have them be labeled with vSIDs. > > With cmdq direct assignment the VM has to issue an ATS invalidation > for each and every physical device using its vSID. There is no HW path > to handle a 1:N v/p SID relationship. > I see, that's for the cmdq assignment. > > Ah, I thought IOMMUFD would be used instead of VFIO_TYPE1*, which should cover > > platform devices (VFIO-platform) or am I missing something? > > It does work with platform, but AFAIK nobody is interested in that so > it hasn't been any focus. Enabling multi-sid nesting, sub stream ids, > etc is some additional work I think. > > But what I mean is the really hard use case for the vSID/pSID mapping > is ATS invalidation and you won't use ATS invalidation on platform so > multi-sid likely doesn't matter. > > STE/CD invalidation could possibly be pushed down through the > per-domain ioctl and replicated to all domain attachments. We don't > have code in the series to do that, but it could work from a uAPI > perspective. > > > If possible, can the UAPI be designed with this in mind, even if not > > implemented now? > > It is reasonable to ask. I think things are extensible enough. I'd > imagine we can add a flag 'secondary ID' and then a new field > 'secondary ID index' to the vdev operations when someone wants to take > this on. > Makes sense, I can take this when I start doing the pKVM work with IOMMUFD, in case it wasn't supported by then. Thanks, Mostafa > Jason
