On Tue, 21 Feb 2017 12:49:18 +0800 Lan Tianyu <tianyu.lan@xxxxxxxxx> wrote: > On 2017年02月21日 04:53, Alex Williamson wrote: > > On Sun, 19 Feb 2017 22:47:06 +0800 > > Lan Tianyu <tianyu.lan@xxxxxxxxx> wrote: > > > >> This patchset proposes a solution to report hardware IOMMU fault > >> event to userspace. Motivation is to make vIOMMU in Qemu get physical > >> fault event and info from IOMMU hardware. vIOMMU is in charge of transforming > >> fault info and inject to guest. The feature is also very important to > >> support first level translation(Translation request with PASID) in VM > >> which requires vIOMMU to inject device page request to VM. > > > > Is the type1 IOMMU backend even a valid target here? vIOMMU support > > with type1 is functional, but not really usable except for very > > specific cases. Type1 is not designed for and not well suited to > > dynamic DMA mapping in a VM, which means that anything other than > > iommu=pt or assigning the device to a VM user or l2 guest is going to > > have horrible performance. Faulting in mappings, as seems to be > > enabled here, sort of implies dynmaic mapping, right? > > Peter's patches have enabled vIOMMU IOVA with assigned device and guest > may change vIOMMU's IOVA page table dynamically. For example, if we > assign NIC and enable DMA protection in VM, NIC driver will dynamically > map/unmap DMA memory when receive/send packages and change vIOMMU IOVA > page table. Yes, it's functional, but does it have sufficient performance to bother to further extend it for the vIOMMU use case? The benefit of the current level of vfio/viommu integration is that we can a) make use of DPDK with assigned devices in a secure mode within the guest, and b) use nested device assignment (though the usefulness of this this versus simple novelty is is questionable). Both of these make use of relatively static mappings through vfio and therefore should not experience much mapping overhead aside from setup and teardown. As I understand your use case, particularly with PASID, you're trying to enable the device to fault in mappings, which implies a dynamic mapping use case. Run a 10GBps NIC (or 40 or 100) in a viommu guest _without_ using iommu=pt. How close do you get to native performance? How close can a virtio device get in the same configuration? What is the usefulness in extending type1 to support piommu faults if it doesn't have worthwhile performance? > > Our container > > based accounting falls apart for type1 backing a vIOMMU as well. Each > > device lives in its own address space and therefore vfio container, > > which means that management of vIOMMU assigned device VMs needs to > > account for locked memory limits per device. This quickly opens a > > large vulnerability if a VM can lock multiples of the VM memory size. > > This seems we have already this issue if we use assigned device with > vIOMMU regardless of whether we report pIOMMU fault event to vIOMMU or > not, right? Yes, so why continue to push viommu features into an iommu model that has these limitations? Perhaps type1 can be extended to avoid them, perhaps a new "type2" iommu backend is a better solution. Either way I don't see how adding iommu fault handling to type1 as it exists today is more than just a proof of concept exercise. > > > > So is what you're developing here just a proof of concept using type1, > > or is this something that you feel actually has value long term? > > We want to pass physical IOMMU fault event to vIOMMU and then inject > virtual fault event to VM. It's necessary to create the channel between > pIOMMU driver and vIOMMU. Currently, IOMMU map/umap requests go through > VFIO IOMMU type1 driver and so I supposed VFIO IOMMU type1 was better > place. This patchset is to show the idea. Another option is to go though > VFIO-PCI driver we have considered. It's very helpful if you can give > some guides here :) I understand, but is this any more than a functional proof of concept? What are your performance requirements? Does type1 meet them? Have you tested? My expectation is that dynamic DMA mapping performance through type1 with a viommu configured for dynamic mappings is abysmal and it's therefore a poor target on which to base additional features such as fault handling. Do you have data to suggest otherwise? Going through the device rather than the container doesn't solve my assertion that type1 is merely functional and does not provide worthwhile performance when used in conjunction with dynamic mappings. Thanks, Alex > >> Introduce two new VFIO cmd for VFIO IOMMU type1 driver > >> - VFIO_IOMMU_SET_FAULT_EVENT_FD > >> Receive eventfd from user space and trigger eventfd when get fault event > >> from IOMMU. > >> > >> - VFIO_IOMMU_GET_FAULT_INFO > >> Return IOMMU fault info to userspace. > >> > >> VFIO IOMMU Type1 driver needs to register IOMMU fault event notifier > >> to IOMMU driver for assigned devices when they are attached to VFIO > >> container. IOMMU driver will run VFIO fault event callback when hardware > >> IOMMU triggers fault events for devices assigned to VFIO container. Then, > >> type1 driver signals eventfd provided by userspace and usrspace gets fault > >> info via new cmd. > >> > >> IOMMU interface are still in design stage and so we still can't register > >> IOMMU fault event notifier to IOMMU driver. This patchset is prototype > >> code to check whether VFIO IOMMU type1 driver is suitable place to deal > >> with IOMMU fault event and just passes build test. > >> > >> Very appreciate for comments. > >> > >> Lan Tianyu (3): > >> VFIO: Add new cmd to receive eventfd from userspace to notify IOMMU > >> fault event > >> VFIO: Add IOMMU fault notifier callback > >> VFIO: Add new cmd for user space to get IOMMU fault info > >> > >> drivers/vfio/vfio_iommu_type1.c | 106 ++++++++++++++++++++++++++++++++++++++++ > >> include/linux/iommu.h | 7 +++ > >> include/uapi/linux/vfio.h | 37 ++++++++++++++ > >> 3 files changed, 150 insertions(+) > >> > > > >
