On Thu, Apr 29, 2021 at 01:04:05PM +1000, David Gibson wrote: > Again, I don't know enough about VDPA to make sense of that. Are we > essentially talking non-PCI virtual devices here? In which case you > could define the VDPA "bus" to always have one-device groups. It is much worse than that. What these non-PCI devices need is for the kernel driver to be part of the IOMMU group of the underlying PCI device but tell VFIO land that "groups don't matter" Today mdev tries to fake this by using singleton iommu groups, but it is really horrible and direcly hacks up the VFIO IOMMU code to understand these special cases. Intel was proposing more special hacking in the VFIO IOMMU code to extend this to PASID. When we get to a /dev/ioasid this is all nonsense. The kernel device driver is going to have to tell drivers/iommu exactly what kind of ioasid it can accept, be it a PASID inside a kernel owned group, a SW emulated 'mdev' ioasid, or whatever. In these cases the "group" idea has become a fiction that just creates a pain. "Just reorganize VDPA to do something insane with the driver core so we can create a dummy group to satisfy an unnecessary uAPI restriction" is not a very compelling argument. So if the nonsensical groups goes away for PASID/mdev, where does it leave the uAPI in other cases? > I don't think simplified-but-wrong is a good goal. The thing about > groups is that if they're there, you can't just "not care" about them, > they affect you whether you like it or not. You really can. If one thing claims the group then all the other group devices become locked out. The main point to understand is that groups are NOT an application restriction! It is a whole system restriction that the operator needs to understand and deal with. This is why things like dpdk don't care about the group at all - there is nothing they can do with the information. If the operator says to run dpdk on a specific device then the operator is the one that has to deal with all the other devices in the group getting locked out. At best the application can make it more obvious that the operator is doing something dangerous, but the current kernel API doesn't seem to really support that either. Jason
