> From: Nicolin Chen <nicolinc@xxxxxxxxxx> > Sent: Saturday, October 26, 2024 7:50 AM > > Following the previous vIOMMU series, this adds another vDEVICE structure, > representing the association from an iommufd_device to an > iommufd_viommu. > This gives the whole architecture a new "v" layer: > > ________________________________________________________________ > _______ > | iommufd (with vIOMMU/vDEVICE) | > | _____________ _____________ | > | | | | | | > | |----------------| vIOMMU |<---| vDEVICE |<------| | > | | | | |_____________| | | > | | ______ | | _____________ ___|____ | > | | | | | | | | | | | > | | | IOAS |<---|(HWPT_PAGING)|<---| HWPT_NESTED |<--| DEVICE | | > | | |______| |_____________| |_____________| |________| | > > |______|________|______________|__________________|_____________ > __|_____| > | | | | | > ______v_____ | ______v_____ ______v_____ ___v__ > | struct | | PFN | (paging) | | (nested) | |struct| > |iommu_device| |------>|iommu_domain|<----|iommu_domain|<---- > |device| > |____________| storage|____________| |____________| |______| > > This vDEVICE object is used to collect and store all vIOMMU-related device > information/attributes in a VM. As an initial series for vDEVICE, add only > the virt_id to the vDEVICE, which is a vIOMMU specific device ID in a VM: > e.g. vSID of ARM SMMUv3, vDeviceID of AMD IOMMU, and vID of Intel VT-d s/vID/vRID/ for VT-d > to > a Context Table. This virt_id helps IOMMU drivers to link the vID to a pID > of the device against the physical IOMMU instance. This is essential for a > vIOMMU-based invalidation, where the request contains a device's vID for a > device cache flush, e.g. ATC invalidation. probably connect this to vCMDQ passthrough? otherwise for sw-based invalidation the userspace can always replace vID with pID before submitting the request. > > Therefore, with this vDEVICE object, support a vIOMMU-based invalidation, > by reusing IOMMUFD_CMD_HWPT_INVALIDATE for a vIOMMU object to > flush cache > with a given driver data. > > As for the implementation of the series, add driver support in ARM SMMUv3 > for a real world use case. > > This series is on Github: > https://github.com/nicolinc/iommufd/commits/iommufd_viommu_p2-v5 > > For testing, try this "with-rmr" branch: > https://github.com/nicolinc/iommufd/commits/iommufd_viommu_p2-v5- > with-rmr > Paring QEMU branch for testing: > https://github.com/nicolinc/qemu/commits/wip/for_iommufd_viommu_p2- > v5 > > Changelog > v5 > * Dropped driver-allocated vDEVICE support > * Changed vdev_to_dev helper to iommufd_viommu_find_dev > v4 > https://lore.kernel.org/all/cover.1729555967.git.nicolinc@xxxxxxxxxx/ > * Added missing brackets in switch-case > * Fixed the unreleased idev refcount issue > * Reworked the iommufd_vdevice_alloc allocator > * Dropped support for IOMMU_VIOMMU_TYPE_DEFAULT > * Added missing TEST_LENGTH and fail_nth coverages > * Added a verification to the driver-allocated vDEVICE object > * Added an iommufd_vdevice_abort for a missing mutex protection > * Added a u64 structure arm_vsmmu_invalidation_cmd for user command > conversion > v3 > https://lore.kernel.org/all/cover.1728491532.git.nicolinc@xxxxxxxxxx/ > * Added Jason's Reviewed-by > * Split this invalidation part out of the part-1 series > * Repurposed VDEV_ID ioctl to a wider vDEVICE structure and ioctl > * Reduced viommu_api functions by allowing drivers to access viommu > and vdevice structure directly > * Dropped vdevs_rwsem by using xa_lock instead > * Dropped arm_smmu_cache_invalidate_user > v2 > https://lore.kernel.org/all/cover.1724776335.git.nicolinc@xxxxxxxxxx/ > * Limited vdev_id to one per idev > * Added a rw_sem to protect the vdev_id list > * Reworked driver-level APIs with proper lockings > * Added a new viommu_api file for IOMMUFD_DRIVER config > * Dropped useless iommu_dev point from the viommu structure > * Added missing index numnbers to new types in the uAPI header > * Dropped IOMMU_VIOMMU_INVALIDATE uAPI; Instead, reuse the HWPT > one > * Reworked mock_viommu_cache_invalidate() using the new iommu helper > * Reordered details of set/unset_vdev_id handlers for proper lockings > v1 > https://lore.kernel.org/all/cover.1723061377.git.nicolinc@xxxxxxxxxx/ > > Thanks! > Nicolin > > Jason Gunthorpe (2): > iommu: Add iommu_copy_struct_from_full_user_array helper > iommu/arm-smmu-v3: Allow ATS for IOMMU_DOMAIN_NESTED > > Nicolin Chen (11): > iommufd/viommu: Add IOMMUFD_OBJ_VDEVICE and > IOMMU_VDEVICE_ALLOC ioctl > iommufd/selftest: Add IOMMU_VDEVICE_ALLOC test coverage > iommu/viommu: Add cache_invalidate to iommufd_viommu_ops > iommufd/hw_pagetable: Enforce invalidation op on vIOMMU-based > hwpt_nested > iommufd: Allow hwpt_id to carry viommu_id for > IOMMU_HWPT_INVALIDATE > iommufd/viommu: Add iommufd_viommu_find_dev helper > iommufd/selftest: Add mock_viommu_cache_invalidate > iommufd/selftest: Add IOMMU_TEST_OP_DEV_CHECK_CACHE test > command > iommufd/selftest: Add vIOMMU coverage for IOMMU_HWPT_INVALIDATE > ioctl > Documentation: userspace-api: iommufd: Update vDEVICE > iommu/arm-smmu-v3: Add arm_vsmmu_cache_invalidate > > drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 9 +- > drivers/iommu/iommufd/iommufd_private.h | 20 ++ > drivers/iommu/iommufd/iommufd_test.h | 30 +++ > include/linux/iommu.h | 48 ++++- > include/linux/iommufd.h | 21 ++ > include/uapi/linux/iommufd.h | 61 +++++- > tools/testing/selftests/iommu/iommufd_utils.h | 83 +++++++ > .../arm/arm-smmu-v3/arm-smmu-v3-iommufd.c | 161 +++++++++++++- > drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 32 ++- > drivers/iommu/iommufd/device.c | 11 + > drivers/iommu/iommufd/driver.c | 13 ++ > drivers/iommu/iommufd/hw_pagetable.c | 36 +++- > drivers/iommu/iommufd/main.c | 7 + > drivers/iommu/iommufd/selftest.c | 98 ++++++++- > drivers/iommu/iommufd/viommu.c | 101 +++++++++ > tools/testing/selftests/iommu/iommufd.c | 204 +++++++++++++++++- > .../selftests/iommu/iommufd_fail_nth.c | 4 + > Documentation/userspace-api/iommufd.rst | 41 +++- > 18 files changed, 942 insertions(+), 38 deletions(-) > > -- > 2.43.0
