On 19/09/2018 00:26, Tian, Kevin wrote: >> If the core actually calls it, we can implement detach_dev :) The >> problem is that the core never calls detach_dev when default_domain is >> present (affects any IOMMU driver that relies on default_domain, >> including AMD), and even in case 4) the default_domain is present for >> the parent device > > Then can we change that core logic so detach_dev is invoked in all > cases? yes there will be some changes in vendor drivers, but I expect > this change trivial (especially considering the gain in IOMMU API > simplicity side as described below). Thinking more about this, there might be a way that doesn't require too much rewriting of IOMMU drivers. When VFIO calls iommu_detach_group to detach a vfio-pci device from its domain, it could do detach_dev(dev, old_domain) followed by attach_dev(dev, default_domain) instead of only attach_dev(dev, default_domain), so: __iommu_attach_group(grp, unmanaged_domain) ops->attach_dev(dev, unmanaged_domain) -> IOMMU driver detaches from default_domain only if the dev isn't in auxiliary mode __iommu_detach_group(grp, unmanaged_domain) if (ops->detach_dev) ops->detach_dev(dev, unmanaged_domain) ops->attach_dev(dev, default_domain) -> IOMMU driver ignores this if still attached to default_domain It's not trivial: since half the IOMMU drivers seem to be using default domain now, their detach_dev callback might have been dead code for a while (I can't find a path where it still gets called), so re-enabling it requires some attention. However, since most of them won't care about aux domains, maybe we could just remove their detach_dev pointer. The resulting logic of attach/detach_dev in IOMMU drivers that do use aux domains still seems unnecessarily convoluted. It could as well be done with separate attach_aux/detach_aux functions. If you want to keep it that way though, I think the above could work for us and I'll try to write an RFC. Thanks, Jean