On 2022-09-06 10:52, Robin Murphy wrote:
On 2022-09-05 15:13, Vasant Hegde wrote:
Steven,
[+Felix, amd-fgx list]
On 9/3/2022 4:29 AM, Steven J Abner wrote:
Hi
I was referred to you from linux-kernel@xxxxxxxxxxxxxxx about the
following issue.
Here is as was written:
On 9/1/22 11:36, Steven J Abner wrote:
Hi
Building a kernel tailored for AMD 2400g on ASRock B450 using
5.18.12 as base.
I stumbled across an odd situation and which lacked Kconfig info and
lead to
oddity.
/drivers/iommu/amd/Kconfig states 'config AMD_IOMMU_V2' is
'tristate' but unlike
many
other tristate configures doesn't mention that module name is
'iommu_v2.ko' and
loading should be done by adding to modules-load.d.
The oddity is that by loading as module is as follows (differences):
builtin iommu_v2 version dmesg:
amdgpu: HMM registered 2048MB device memory
amdgpu: Topology: Add APU node [0x0:0x0]
amdgpu: Topology: Add APU node [0x15dd:0x1002]
AMD-Vi: AMD IOMMUv2 loaded and initialized
kfd kfd: amdgpu: added device 1002:15dd
kfd kfd: amdgpu: Allocated 3969056 bytes on gart
memmap_init_zone_device initialised 524288 pages in 0ms
IOMMU V2 modules provides IOMMU feature like attaching device to
process. I think amdgpu uses those features if available.
So in this case amdgpu is using those IOMMU features.
module not loaded due to missing iommu.conf dmesg:
amdgpu: CRAT table disabled by module option
amdgpu: Topology: Add CPU node
amdgpu: Virtual CRAT table created for CPU
kfd kfd: amdgpu: GC IP 090100 not supported in kfd
module load through iommu.conf dmesg:
amdgpu: CRAT table disabled by module option
amdgpu: Topology: Add CPU node
amdgpu: Virtual CRAT table created for CPU
AMD-Vi: AMD IOMMUv2 loaded and initialized
kfd kfd: amdgpu: GC IP 090100 not supported in kfd
Note, only difference on witk/without iommu.conf is:
AMD-Vi: AMD IOMMUv2 loaded and initialized
I think in this case iommu_v2.ko module got loaded after GPU
initialized. Hence amdgpu is not using iommu v2 features.
So does this mean missing features by not having builtin?
If not, should Kconfig have hint about module and loading?
@Felix,
I see that drivers/gpu/drm/amd/amdkfd/Kconfig contains below line
imply AMD_IOMMU_V2 if X86_64
Should we change `s/imply/select` ?
"select" might help when KFD is built-in, but it probably still wants
a MODULE_SOFTDEP() to enforce load order when they're both modular.
We don't want to make it a hard dependency because only a small subset
of all AMD GPUs use iommuv2, and we have fallbacks in place for even
those to work without it.
Regards,
Felix
Robin.
