Doorbells in AMDGPU drivers are currently managed by different clients in a scattered way, across may places. The existing clients are: - AMDGPU graphics driver for kernel level doorbell writes. - AMDGPU MES module for kernel level doorbell write (MES ring test and aggregated doorbells). - AMDGPU MES module for MES process doorbells. - AMDKFD module for KFD/KIQ kernel doorbells. - AMDKFD module for KFD process doorbells. - AMDGPU usermode queues for usermode doorbell writes (upcoming). This patch series introduces doorbell-manager to keep the doorbell handling at a central place. The fundamental changes are: - Introduce and accommodate a new GEM domain for doorbells. - Prepare the AMDGPU ttm backend for handling doorbell allocation. - Create doorbell BOs for kernel-level and process level doorbell opertations, and place it in existing structures. - Modify the existing graphics, KFD and MES code to use the doorbell-manager functions. - Remove the existing doorbell management code in KFD/MES. The idea is that: - a kernel client can call doorbell manager functions to allocate/free doorbell pages. - a usermode app can directly allocate a page from the doorbell bar just like a GEM object and use it for different usermode queues. - There is no direct UAPI change in this series, just an additional flag for GEM_OBJECT_DOORBELL, here: https://gitlab.freedesktop.org/mesa/drm/-/merge_requests/286 PS: This series has been sanity tested with kfd_test_suit to ensure it is not introducing any regressions due to kfd doorbell changes. Alex Deucher (2): drm/amdgpu: add UAPI for allocating doorbell memory drm/amdgpu: accommodate DOMAIN/PL_DOORBELL Shashank Sharma (10): drm/amdgpu: create a new file for doorbell manager drm/amdgpu: don't modify num_doorbells for mes drm/amdgpu: initialize ttm for doorbells drm/amdgpu: create kernel doorbell pages get absolute offset from doorbell index drm/amdgpu: use doorbell manager for kfd kernel doorbells drm/amdgpu: use doorbell manager for kfd process doorbells drm/amdgpu: remove unused functions and variables drm/amdgpu: use doorbell mgr for MES kernel doorbells drm/amdgpu: cleanup MES process level doorbells drivers/gpu/drm/amd/amdgpu/Makefile | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 174 +------------ drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell.h | 17 +- .../gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c | 230 ++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_mes.c | 148 +++-------- drivers/gpu/drm/amd/amdgpu/amdgpu_mes.h | 16 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 11 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 2 + .../gpu/drm/amd/amdgpu/amdgpu_res_cursor.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 30 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h | 1 + drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 13 - drivers/gpu/drm/amd/amdkfd/kfd_device.c | 2 - .../drm/amd/amdkfd/kfd_device_queue_manager.c | 8 +- drivers/gpu/drm/amd/amdkfd/kfd_doorbell.c | 207 +++++----------- drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 27 +- drivers/gpu/drm/amd/amdkfd/kfd_process.c | 21 +- .../amd/amdkfd/kfd_process_queue_manager.c | 22 +- include/uapi/drm/amdgpu_drm.h | 7 +- 19 files changed, 446 insertions(+), 494 deletions(-) create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_doorbell_mgr.c -- 2.40.0
