On Fri, Apr 29, 2022 at 04:22:56PM +1000, David Gibson wrote: > On Fri, Apr 29, 2022 at 01:21:30AM +0000, Tian, Kevin wrote: > > > From: Jason Gunthorpe <jgg@xxxxxxxxxx> > > > Sent: Thursday, April 28, 2022 11:11 PM > > > > > > > > > > 3) "dynamic DMA windows" (DDW). The IBM IOMMU hardware allows for > > > 2 IOVA > > > > windows, which aren't contiguous with each other. The base addresses > > > > of each of these are fixed, but the size of each window, the pagesize > > > > (i.e. granularity) of each window and the number of levels in the > > > > IOMMU pagetable are runtime configurable. Because it's true in the > > > > hardware, it's also true of the vIOMMU interface defined by the IBM > > > > hypervisor (and adpoted by KVM as well). So, guests can request > > > > changes in how these windows are handled. Typical Linux guests will > > > > use the "low" window (IOVA 0..2GiB) dynamically, and the high window > > > > (IOVA 1<<60..???) to map all of RAM. However, as a hypervisor we > > > > can't count on that; the guest can use them however it wants. > > > > > > As part of nesting iommufd will have a 'create iommu_domain using > > > iommu driver specific data' primitive. > > > > > > The driver specific data for PPC can include a description of these > > > windows so the PPC specific qemu driver can issue this new ioctl > > > using the information provided by the guest. > > > > > > The main issue is that internally to the iommu subsystem the > > > iommu_domain aperture is assumed to be a single window. This kAPI will > > > have to be improved to model the PPC multi-window iommu_domain. > > > > > > > From the point of nesting probably each window can be a separate > > domain then the existing aperture should still work? > > Maybe. There might be several different ways to represent it, but the > vital piece is that any individual device (well, group, technically) > must atomically join/leave both windows at once. I'm not keen on the multi-iommu_domains because it means we have to create the idea that a device can be attached to multiple iommu_domains, which we don't have at all today. Since iommu_domain allows PPC to implement its special rules, like the atomicness above. Jason
