The motivation for this series comes from pending UMD submission work by AMD [1], ARM [3], and the Xe team, who are also beginning to look at this. Sima has suggested [4] some common driver preemptive fences and semantics, which we all agree on. This is the first attempt to implement them, based on Xe's existing long-running preemptive fences. The semantics are fairly simple: preemptive fences attached to a dma-resv must wait to enable signaling (and thus preempt device execution) until all fences in other slots have been signaled. These semantics are necessary to avoid deadlocks when preempting a device while a user submission is pending, and resuming device execution depends on the user submission completing. The semantics align with Christian's view [5], which I also arrived at independently (with a little help from Sima). Implemented via the new dma-resv slot DMA_RESV_USAGE_PREEMPT, a common embedded base preemptive fence class with driver operations, and updates to the scheduler to adhere to these semantics. The current Xe long-running preemptive fences have been converted to the new common code, and UMD submission is expected to follow (hopefully) shortly thereafter in a subsequent series. Not intended to be presented as the final solution, but rather to kickstart serious discussions on this topic. Matt [1] https://patchwork.freedesktop.org/series/113675/ [2] https://patchwork.freedesktop.org/series/114385/ [3] https://patchwork.freedesktop.org/series/137924/ [4] https://patchwork.kernel.org/project/dri-devel/cover/20240828172605.19176-1-mihail.atanassov@xxxxxxx/#26011577 [5] https://patchwork.kernel.org/project/dri-devel/cover/20240828172605.19176-1-mihail.atanassov@xxxxxxx/#26011853 Matthew Brost (6): dma-resv: Add DMA_RESV_USAGE_PREEMPT drm/sched: Teach scheduler about DMA_RESV_USAGE_PREEMPT dma-fence: Add dma_fence_preempt base class drm/sched: Teach scheduler about dma_fence_prempt type drm/xe: Use DMA_RESV_USAGE_PREEMPT for preempt fences drm/xe: Use dma_fence_preempt base class drivers/dma-buf/Makefile | 2 +- drivers/dma-buf/dma-fence-preempt.c | 102 ++++++++++++++++++++ drivers/dma-buf/dma-resv.c | 43 ++++++--- drivers/dma-buf/st-dma-resv.c | 2 +- drivers/gpu/drm/scheduler/sched_entity.c | 29 +++++- drivers/gpu/drm/scheduler/sched_main.c | 50 +++++++--- drivers/gpu/drm/xe/xe_bo.c | 22 +---- drivers/gpu/drm/xe/xe_guc_submit.c | 3 + drivers/gpu/drm/xe/xe_hw_engine_group.c | 4 +- drivers/gpu/drm/xe/xe_migrate.c | 4 +- drivers/gpu/drm/xe/xe_preempt_fence.c | 81 +++++----------- drivers/gpu/drm/xe/xe_preempt_fence.h | 2 +- drivers/gpu/drm/xe/xe_preempt_fence_types.h | 11 +-- drivers/gpu/drm/xe/xe_pt.c | 12 +-- drivers/gpu/drm/xe/xe_vm.c | 22 +---- include/drm/gpu_scheduler.h | 15 +++ include/linux/dma-fence-preempt.h | 54 +++++++++++ include/linux/dma-fence.h | 1 + include/linux/dma-resv.h | 24 +++-- 19 files changed, 326 insertions(+), 157 deletions(-) create mode 100644 drivers/dma-buf/dma-fence-preempt.c create mode 100644 include/linux/dma-fence-preempt.h -- 2.34.1
