Am 2022-02-18 um 11:53 schrieb Alex Sierra:
This parameter controls xGMI p2p communication, which is enabled by
default. However, it can be disabled by setting it to 0. In case xGMI
p2p is disabled in a dGPU, PCIe p2p interface will be used instead.
This parameter is ignored in GPUs that do not support xGMI
p2p configuration.
Signed-off-by: Alex Sierra <alex.sierra@xxxxxxx>
Acked-by: Luben Tuikov <luben.tuikov@xxxxxxx>
Acked-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@xxxxxxx>
---
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 8 ++++++++
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 4 ++--
3 files changed, 11 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index f97848a0ed14..7e95d8bd2338 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -217,6 +217,7 @@ extern int amdgpu_mes;
extern int amdgpu_noretry;
extern int amdgpu_force_asic_type;
extern int amdgpu_smartshift_bias;
+extern int amdgpu_use_xgmi_p2p;
#ifdef CONFIG_HSA_AMD
extern int sched_policy;
extern bool debug_evictions;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 2f8eafb6cf22..6156265f3178 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -181,6 +181,7 @@ int amdgpu_tmz = -1; /* auto */
int amdgpu_reset_method = -1; /* auto */
int amdgpu_num_kcq = -1;
int amdgpu_smartshift_bias;
+int amdgpu_use_xgmi_p2p = -1;
static void amdgpu_drv_delayed_reset_work_handler(struct work_struct *work);
@@ -677,6 +678,13 @@ MODULE_PARM_DESC(force_asic_type,
"A non negative value used to specify the asic type for all supported GPUs");
module_param_named(force_asic_type, amdgpu_force_asic_type, int, 0444);
+/**
+ * DOC: use_xgmi_p2p (int)
+ * Enables/disables XGMI P2P interface (0 = disable, 1 = enable). The Default is -1 (enabled).
+ */
+MODULE_PARM_DESC(use_xgmi_p2p,
+ "Disable XGMI P2P interface (0 = disable; 1 = enable; -1 default, enabled)");
+module_param_named(use_xgmi_p2p, amdgpu_use_xgmi_p2p, int, 0444);
#ifdef CONFIG_HSA_AMD
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 207cd01435b5..bbe1bac61f15 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -2330,8 +2330,8 @@ struct amdgpu_bo_va *amdgpu_vm_bo_add(struct amdgpu_device *adev,
#else
dma_resv_assert_held(amdkcl_ttm_resvp(&vm->root.bo->tbo));
#endif
-
- if (amdgpu_dmabuf_is_xgmi_accessible(adev, bo)) {
+ if (amdgpu_use_xgmi_p2p &&
+ amdgpu_dmabuf_is_xgmi_accessible(adev, bo)) {
This does nothing to tell user mode that XGMI is not available. We need
to update the topology information as well. For that, this check needs
to be applied at a lower level. I think amdgpu_xgmi_same_hive would be
the right place. If amdgpu_use_xgmi_p2p is set, amdgpu_xgmi_same_hive
should return false.
Regards,
Felix
bo_va->is_xgmi = true;
/* Power up XGMI if it can be potentially used */
amdgpu_xgmi_set_pstate(adev, AMDGPU_XGMI_PSTATE_MAX_VEGA20);
