On Wed, Aug 24, 2022 at 04:02:01PM -0600, Alex Williamson wrote: > On Tue, 23 Aug 2022 21:38:08 -0300 > Jason Gunthorpe <jgg@xxxxxxxxxx> wrote: > > So, I would prefer to comment it like this and if someone comes with a > > driver that wants to use it in some other way they have to address > > these problems so it works generally and correctly. I don't want to > > write more code to protect against something that auditing tells us > > doesn't happen today. > > > > The whole thing should naturally become fixed fairly soon, as once we > > have Yishai and Joao's changes there will be no use for the dirty > > tracking code in type1 that is causing this problem. > > I assume this is referring to the DMA logging interface and > implementation for mlx5[1], but I don't see anyone else getting on board > with that proposal (or reviewing). We have a large group pushing in this direction together. Yishai implemented the API in vfio for mlx5 showing how a device centric tracker and work. In my research I found no other groups besides NVIDIA planning to do device tracking, so it is not surprising his series has been as well reviewed as it probably will be. The other use cases inside NVIDIA are also happy with this design. Joao implemented the similar API in iommufd for the system iommu, and did this with hisilicon's support because they need it for their merged hisilicon's vfio driver. https://github.com/jpemartins/linux/commit/7baa3d51417419690aa8b57f6cc58be7ab3a40a9 I understand Intel to use this with IDXD as well.. So, basically everyone who is working in the migratable VFIO device space is on board with, and contributing to, this plan right now. > IOMMU DMA logging, so the above seems a bit of a bold claim. Do we > expect system IOMMU logging flowing through this device feature or will > there be an interface at the shared IOMMU context level? Joao's draft series shows the basic idea: https://github.com/jpemartins/linux/commit/fada208468303a3a6df340ac20cda8fc1b869e7a#diff-4318a59849ffa0d4aa992221863be15cde4c01e1f82d2f6d06337cfe6dd316afR228 We are working this all through in pieces as they become ready. > Do we expect mdev drivers supporting migration to track their dirty > iovas locally and implement this feature? Assuming we ever get a SW only device that can do migration.. I would expect there to be a library to manage the dirty bitmap datastructure and the instance of that datastructure to be linked to the IOAS the device is attached to. Ie one IOAS one datastructure. The additional code in the mdev driver would be a handful of lines. If we want it to report out through the vfio or through a iommufd API is an interesting question I don't have a solid answer to right now. Either will work - iommufd has a nice hole to put a "sw page table" object parallel to the "hw page table" object that the draft series put the API on. The appeal is it makes the sharing of the datastructure clearer to userpsace. The trouble with mlx5 is there isn't a nice iommufd spot to put a device tracker. The closest is on the device id, but using the device id in APIs intended for "page table" id's is wonky. And there is no datatructure sharing here, obviously. Not such a nice fit. Jason
