> From: Jason Gunthorpe <jgg@xxxxxxxxxx> > Sent: Thursday, May 5, 2022 10:08 PM > > On Thu, May 05, 2022 at 07:40:37AM +0000, Tian, Kevin wrote: > > > In concept this is an iommu property instead of a domain property. > > Not really, domains shouldn't be changing behaviors once they are > created. If a domain supports dirty tracking and I attach a new device > then it still must support dirty tracking. That sort of suggests that userspace should specify whether a domain supports dirty tracking when it's created. But how does userspace know that it should create the domain in this way in the first place? live migration is triggered on demand and it may not happen in the lifetime of a VM. and if the user always creates domain to allow dirty tracking by default, how does it know a failed attach is due to missing dirty tracking support by the IOMMU and then creates another domain which disables dirty tracking and retry-attach again? In any case IMHO having a device capability still sounds applausive even in above model so userspace can create domain with right property based on a potential list of devices to be attached. Once the domain is created, then further attached devices must be compatible to the domain property. > > I suppose we may need something here because we need to control when > domains are re-used if they don't have the right properties in case > the system iommu's are discontiguous somehow. > > ie iommufd should be able to assert that dirty tracking is desired and > an existing non-dirty tracking capable domain will not be > automatically re-used. > > We don't really have the right infrastructure to do this currently. > > > From this angle IMHO it's more reasonable to report this IOMMU > > property to userspace via a device capability. If all devices attached > > to a hwpt claim IOMMU dirty tracking capability, the user can call > > set_dirty_tracking() on the hwpt object. > > Inherent domain properties need to be immutable or, at least one-way, > like enforced coherent, or it just all stops making any kind of sense. > > > Once dirty tracking is enabled on a hwpt, further attaching a device > > which doesn't claim this capability is simply rejected. > > It would be OK to do as enforced coherent does as flip a domain > permanently into dirty-tracking enabled, or specify a flag at domain > creation time. > Either way I think a device capability is useful for the user to decide the necessity of flipping one-way or specifying a flag at domain creation. Thanks Kevin
