On Tue, Jan 30, 2018 at 6:53 PM, Felix Kuehling <felix.kuehling at amd.com> wrote: > > On 2018-01-30 08:59 AM, Christian König wrote: >> Am 29.01.2018 um 23:08 schrieb Felix Kuehling: >>> On 2018-01-26 03:13 PM, Christian König wrote: >>>> [SNIP] >>>> +#ifdef CONFIG_DRM_AMDGPU_ATC >>>> + r = amd_iommu_init_device(adev->pdev, 0x10000); >>> KFD queries how many PASIDs the IOMMU can support with >>> amd_iommu_device_info. KFD only assigns PASIDs within that range. It can >>> be much smaller than the 16-bits supported by the GPU. >>> >>> For a VM that uses ATC, you need to make sure it gets a PASID in the >>> range supported by the IOMMU. The PASID manager already supports that >>> and keeps smaller PASIDs for users that really need them. >> >> Yeah, seen that and I'm not really keen about it. >> >> Especially since we need multiple types of PASIDs here: >> 1. For GPUVM debugging and HMM faults, where we can use the full 16bit >> range without worrying about what IOMMU can do. >> 2. For ATC use case where we need to keep the IOMMU in the picture. >> >> Are there any hardware limitations which blocks us from using a per >> device PASID? That would simplify the whole handling quite a bit. > > Conceptually PASIDs are device-specific process IDs. KFD uses the same > PASID on all devices, but that's not necessary. > > Currently you allocate PASIDs per VM or per open device file. So you'll > end up with different PASIDs on each device. I think you can even end up > with multiple PASIDs in each process even on the same device, if you > create multiple VMs. That doesn't seem to be a problem for the IOMMU driver. > >> >> Additional to that we don't really want this direct relationship >> between amdgpu/amdkfd and the amd_iommu_v2 driver. > > Not sure what you mean. Right now amdgpu and amdkfd don't coordinate > their PASIDs (other than using a common allocator to avoid using the > same PASID for different things). > >> >> So what do you think about moving the PASID handling into the IOMMU >> driver? And abstracting which driver is in use through the iommu_ops? > > What if both drivers want to use the IOMMU? > There was some discussion last year about a generic PASID allocator in the iommu subsystem: https://lists.linuxfoundation.org/pipermail/iommu/2017-June/022159.html Unfortunately, I don't think it got any further then that. Oded > Regards, > Felix > >> >> Regards, >> Christian. >> >>> >>> Regards, >>> Felix > > _______________________________________________ > amd-gfx mailing list > amd-gfx at lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
