Am 2022-08-28 um 11:28 schrieb Christian König:
Am 26.08.22 um 23:49 schrieb Felix Kuehling:
On 2022-08-26 11:47, Alex Sierra wrote:
[Why] Devices with CPU XGMI iolink do not support PCIe peer access.
Signed-off-by: Alex Sierra <alex.sierra@xxxxxxx>
---
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index ce7d117efdb5..1ff66718639d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -5538,7 +5538,8 @@ bool amdgpu_device_is_peer_accessible(struct
amdgpu_device *adev,
return pcie_p2p && p2p_access && (adev->gmc.visible_vram_size &&
adev->gmc.real_vram_size == adev->gmc.visible_vram_size &&
!(adev->gmc.aper_base & address_mask ||
- aper_limit & address_mask));
+ aper_limit & address_mask) &&
+ !adev->gmc.xgmi.connected_to_cpu);
Maybe it would be more readable if you added this to the
initialization of p2p_access. In that case you can also save yourself
the call to pci_p2pdma_distance_many and all the aperture checks.
Like this:
bool p2p_access = !adev->gmc.xgmi.connected_to_cpu &&
!(pci_p2pdma_distance_many(adev->pdev, &peer_adev->dev, 1,
true) < 0);
Yeah, agree. Static checks should come first.
But another question: When we have an XGMI link to the CPU all local
memory is directly accessible, isn't it?
So the PCIe root complex should be able to allow PCIe device to access
our local memory, even with snooping.
Or do I miss something here?
This patch is meant for the case that the module parameter
amdgpu_use_xgmi_p2p=0. With XGMI disabled, I don't think PCIe P2P would
work on this platform. This patch will ensure that we don't report PCIe
P2P links in the KFD topology, and don't attempt to create PCIe P2P
mappings and potentially DMA mappings in the page tables.
Regards,
Felix
Regards,
Christian.
Regards,
Felix
#else
return false;
#endif
