On Mon, May 16, 2022 at 05:59:01PM -0400, Matthew Rosato wrote: > On 5/16/22 2:35 PM, Jason Gunthorpe wrote: > > On Mon, May 16, 2022 at 02:30:46PM -0400, Matthew Rosato wrote: > > > > > Conceptually I think this would work for QEMU anyway (it always sets the kvm > > > before we open the device). I tried to test the idea quickly but couldn't > > > get the following to apply on vfio-next or your vfio_group_locking -- but I > > > understand what you're trying to do so I'll re-work and try it out. > > > > I created it on 8c9350e9bf43de1ebab3cc8a80703671e6495ab4 which is the > > vfio_group_locking.. I can send you a github if it helps > > https://github.com/jgunthorpe/linux/commits/vfio_group_lockin > > > Thanks -- I was able to successfully test your proposed idea (+ some changes > to make it compile :)) on top of vfio_group_locking along with a modified > version of my zdev series. I also tried it out with vfio-ap successfully, > but have nothing to test GVT with. > > That said, this has caused me to realize that 'iommu: > iommu_group_claim_dma_owner() must always assign a domain' breaks s390x > vfio-pci :( I wonder if it is due to the way s390x PCI currently switches > between dma ops and iommu ops. It looks like it breaks vfio-ap mdevs too, > but I know less about that -- I will have to investigate both more > tomorrow. At the very least it appears that s390 thinks that attach/detach_dev are strictly paired and that isn't how the iommu ops are defined. attach_dev can be called multiple times without any detach_dev. Possibly the unbalance of zpci_register_ioat / zpci_unregister_ioat causes the trouble? Jason
