On Thu, 12 Sept 2024 at 12:29, Baolu Lu <baolu.lu@xxxxxxxxxxxxxxx> wrote: > > On 9/12/24 11:42 AM, Zhangfei Gao wrote: > > On Wed, 28 Aug 2024 at 05:32, Nicolin Chen<nicolinc@xxxxxxxxxx> wrote: > >> On Tue, Aug 27, 2024 at 12:51:30PM -0300, Jason Gunthorpe wrote: > >>> This brings support for the IOMMFD ioctls: > >>> > >>> - IOMMU_GET_HW_INFO > >>> - IOMMU_HWPT_ALLOC_NEST_PARENT > >>> - IOMMU_DOMAIN_NESTED > >>> - ops->enforce_cache_coherency() > >>> > >>> This is quite straightforward as the nested STE can just be built in the > >>> special NESTED domain op and fed through the generic update machinery. > >>> > >>> The design allows the user provided STE fragment to control several > >>> aspects of the translation, including putting the STE into a "virtual > >>> bypass" or a aborting state. This duplicates functionality available by > >>> other means, but it allows trivially preserving the VMID in the STE as we > >>> eventually move towards the VIOMMU owning the VMID. > >>> > >>> Nesting support requires the system to either support S2FWB or the > >>> stronger CANWBS ACPI flag. This is to ensure the VM cannot bypass the > >>> cache and view incoherent data, currently VFIO lacks any cache flushing > >>> that would make this safe. > >>> > >>> Yan has a series to add some of the needed infrastructure for VFIO cache > >>> flushing here: > >>> > >>> https://lore.kernel.org/linux-iommu/20240507061802.20184-1-yan.y.zhao@xxxxxxxxx/ > >>> > >>> Which may someday allow relaxing this further. > >>> > >>> Remove VFIO_TYPE1_NESTING_IOMMU since it was never used and superseded by > >>> this. > >>> > >>> This is the first series in what will be several to complete nesting > >>> support. At least: > >>> - IOMMU_RESV_SW_MSI related fixups > >>> https://lore.kernel.org/linux-iommu/cover.1722644866.git.nicolinc@xxxxxxxxxx/ > >>> - VIOMMU object support to allow ATS and CD invalidations > >>> https://lore.kernel.org/linux-iommu/cover.1723061377.git.nicolinc@xxxxxxxxxx/ > >>> - vCMDQ hypervisor support for direct invalidation queue assignment > >>> https://lore.kernel.org/linux-iommu/cover.1712978212.git.nicolinc@xxxxxxxxxx/ > >>> - KVM pinned VMID using VIOMMU for vBTM > >>> https://lore.kernel.org/linux-iommu/20240208151837.35068-1-shameerali.kolothum.thodi@xxxxxxxxxx/ > >>> - Cross instance S2 sharing > >>> - Virtual Machine Structure using VIOMMU (for vMPAM?) > >>> - Fault forwarding support through IOMMUFD's fault fd for vSVA > >>> > >>> The VIOMMU series is essential to allow the invalidations to be processed > >>> for the CD as well. > >>> > >>> It is enough to allow qemu work to progress. > >>> > >>> This is on github:https://github.com/jgunthorpe/linux/commits/smmuv3_nesting > >>> > >>> v2: > >> As mentioned above, the VIOMMU series would be required to test > >> the entire nesting feature, which now has a v2 rebasing on this > >> series. I tested it with a paring QEMU branch. Please refer to: > >> https://lore.kernel.org/linux-iommu/cover.1724776335.git.nicolinc@xxxxxxxxxx/ > >> Also, there is another new VIRQ series on top of the VIOMMU one > >> and this nesting series. And I tested it too. Please refer to: > >> https://lore.kernel.org/linux-iommu/cover.1724777091.git.nicolinc@xxxxxxxxxx/ > >> > >> With that, > >> > >> Tested-by: Nicolin Chen<nicolinc@xxxxxxxxxx> > >> > > Have you tested the user page fault? > > > > I got an issue, when a user page fault happens, > > group->attach_handle = iommu_attach_handle_get(pasid) > > return NULL. > > > > A bit confused here, only find IOMMU_NO_PASID is used when attaching > > > > __fault_domain_replace_dev > > ret = iommu_replace_group_handle(idev->igroup->group, hwpt->domain, > > &handle->handle); > > curr = xa_store(&group->pasid_array, IOMMU_NO_PASID, handle, GFP_KERNEL); > > > > not find where the code attach user pasid with the attach_handle. > > Have you set iommu_ops::user_pasid_table for SMMUv3 driver? Thanks Baolu, Nico Yes, after arm_smmu_ops = { + .user_pasid_table = 1, find_fault_handler can go inside attach_handle = iommu_attach_handle_get(IOMMU_NO_PASID); qemu handler also gets called. But hardware reports errors and needs reset, still in check. [Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 0 Thanks
