Am 2022-05-31 um 13:02 schrieb Ramesh Errabolu:
Extend current kernel config requirements of amdgpu by adding config HSA_AMD_P2P. Enabling HSA_AMD_P2P is REQUIRED to support peer-to-peer communication, in both data and control planes, among AMD GPU devices that are connected via PCIe and have large BAR vBIOS Signed-off-by: Ramesh Errabolu <Ramesh.Errabolu@xxxxxxx> --- drivers/gpu/drm/amd/amdkfd/Kconfig | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/gpu/drm/amd/amdkfd/Kconfig b/drivers/gpu/drm/amd/amdkfd/Kconfig index 8cc0a76ddf9f..26614f5f20ea 100644 --- a/drivers/gpu/drm/amd/amdkfd/Kconfig +++ b/drivers/gpu/drm/amd/amdkfd/Kconfig @@ -25,3 +25,11 @@ config HSA_AMD_SVM preemptions and one based on page faults. To enable page fault based memory management on most GFXv9 GPUs, set the module parameter amdgpu.noretry=0. + +config HSA_AMD_P2P + bool "HSA kernel driver support for peer-to-peer for AMD GPU devices" + depends on HSA_AMD && PCI_P2PDMA && DMABUF_MOVE_NOTIFY + help + Enable this if you want to access AMD GPU peer devices, in both data + and control planes, that are connected via PCIe and have large BAR vBIOS
I have not seen the terms "data plane" and "control plane" used in the context of GPUs. As far as I can tell, this terminology is more common in the context network routing. I think it could cause confusion to introduce these terms without an explanation to users.
The sentence "... if you want to access AMD GPU peer devices ..." seems to address someone writing an application. This help message is meant for users and admins building a kernel, who may want to run compute applications, not for compute application developers.
I would also not mention large-BAR VBIOSes because the BAR can often be resized even with a small-BAR VBIOS.
Therefore I would recommend an alternative text here that avoids uncommon terminology and addresses the concerns of users rather than application developers:
Enable peer-to-peer (P2P) communication between AMD GPUs over the PCIe bus. This can improve performance of multi-GPU compute applications and libraries by enabling GPUs to access data directly in peer GPUs' memory without intermediate copies in system memory. This P2P feature is only enabled on compatible chipsets, and between GPUs with large memory BARs that expose the entire VRAM in PCI bus address space within the physical address limits of the GPUs. Regards, Felix
+
