On Thu, Apr 13, 2023 at 11:34:17AM +0100, Sarah Walker wrote: > This patch adds the initial DRM driver for Imagination Technologies PowerVR > GPUs, starting with those based on our Rogue architecture. It's worth pointing > out that this is a new driver, written from the ground up, rather than a > refactored version of our existing downstream driver (pvrsrvkm). > > This new DRM driver supports: > - GEM shmem allocations > - dma-buf / PRIME > - Per-context (device node open) userspace managed virtual address space > - DRM sync objects (binary and timeline) > - Power management suspend / resume > - GPU job submission (geometry, fragment, compute, transfer) > - META firmware processor > - MIPS firmware processor > - Basic GPU hang recovery > > Still to do: > - Document format of job streams and static context state > - GPU hang detection Please use drm/sched for this. There's a few reasons: - linux dma_fence rules are extremely strict, there's pretty much zero wiggle room in semantics. Depending upon how your hardware works it should still be possible to at least resolve some of the dma_fence cpu waits into fw waits if you want that (maybe just within a vkDevice, maybe even across processes), but I really don't want 6 different versions of "how to convert fw semaphores into dma_fence and get it subtle wrong" in upstream. - linux dma_fence is very much a linux desktop (includes cros) concept only. Windows doesn't need this, android doesn't, server hpc (not sure that'll be a thing for pvr?) doesn't need this. Hard-worn experience is that linux desktop absolutely needs to look like windows/android to the gpu or things wont work well enough. If we handle the dma_fence rules in shared software (because it really is just a software construct), then we get dma_fence compat while driving the gpu 100% like anything else would. I haven't checked pvr but some driver teams have been floating insteresting secure fw semaphore plans where you'd kinda get something that looks like a dma_fence, but in the details it's not. Also the other reason for drm/sched is shared naming and structures. We have a lot of "fw-sched, aimed for vk" drivers in the queue for upstreaming, and if they all invent their complete vendor-specific inhouse concepts, then drivers defacto become unreadable for anyone else. So they need to be infused with at least the shared concepts so that I can git grep drm_sched and find the pieces I want to look at, and not be forced to read the entire driver. Related to this I think all vk-first drivers should also adopt the under-discussion gpuva/vm_bind/whatever you call it shared concepts. What exactly that looks like is still tbd I think, but again some minimal shared structs/concepts in this space, while retaining all the flexibility drivers need, is called for. I think aside from these two areas going full hw specific is all fine. Uapi review is all up to mesa folks imo (and the consensus reached&documented around vm_bind or whatever it's going to be called). For the driver itself just a few comments: - you use devm_ already, also using drmm_ is probably a good idea. - the hand-rolled pvr_gem.c does not look good, I think we're way past the point where drivers hand-rolling entire gem from nothing is ready for upstream. - I think it would be really good to use the dma_fence signalling critical section annotations (I plan to resend the ones I have for drm/sched so that they're opt-out for current drivers, but not for new ones). From experience you can savely assume you got things wrong without these :-) Cheers, Daniel > - Handling for running out of parameter buffer space (this has made progress > since RFC v1, but is still incomplete) > > Currently our main focus is on our GX6250, AXE-1-16M and BXS-4-64 GPUs. Testing > so far has been done using an Acer Chromebook R13 (GX6250 GPU) and a TI SK-AM62 > board (AXE-1-16M GPU). An example SK-AM62 development board can be obtained > here: https://beagleboard.org/play > > Firmware for the GX6250 and AXE-1-16M can be found here: > https://gitlab.freedesktop.org/frankbinns/linux-firmware/-/tree/powervr > > A Vulkan driver that works with our downstream kernel driver has already been > merged into Mesa [1][2]. Support for this new DRM driver is being maintained in > a draft merge request [3], with the branch located here: > https://gitlab.freedesktop.org/frankbinns/mesa/-/tree/powervr-winsys > > The Vulkan driver is progressing towards Vulkan 1.0. We're code complete, and > are working towards passing conformance. This is a work in progress, so we're > not sharing numbers just yet. > > We are still looking at whether any functionality can be replaced with DRM > common code. We are still considering drm_sched and the GPU VA manager. > > The code in this patch, along with some of its history, can also be found here: > https://gitlab.freedesktop.org/frankbinns/powervr/-/tree/powervr-next > > Sending this out now as it felt like a good point to get some feedback. > > [1] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15243 > [2] https://gitlab.freedesktop.org/mesa/mesa/-/tree/main/src/imagination/vulkan > [3] https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15507 > > v2: > * Redesigned and simplified UAPI based on RFC feedback from XDC 2022 > * Support for transfer and partial render jobs > * Support for timeline sync objects > > RFC v1: https://lists.freedesktop.org/archives/dri-devel/2022-August/367814.html > > Sarah Walker (2): > drm/imagination: Add initial Imagination Technologies PowerVR driver > dt-bindings: gpu: Add Imagination Technologies PowerVR GPU > > .../devicetree/bindings/gpu/img,powervr.yaml | 105 + > drivers/gpu/drm/imagination/Kconfig | 11 + > drivers/gpu/drm/imagination/Makefile | 36 + > drivers/gpu/drm/imagination/pvr_ccb.c | 380 + > drivers/gpu/drm/imagination/pvr_ccb.h | 51 + > drivers/gpu/drm/imagination/pvr_cccb.c | 390 ++ > drivers/gpu/drm/imagination/pvr_cccb.h | 112 + > drivers/gpu/drm/imagination/pvr_context.c | 1428 ++++ > drivers/gpu/drm/imagination/pvr_context.h | 412 ++ > drivers/gpu/drm/imagination/pvr_debugfs.c | 53 + > drivers/gpu/drm/imagination/pvr_debugfs.h | 29 + > drivers/gpu/drm/imagination/pvr_device.c | 762 ++ > drivers/gpu/drm/imagination/pvr_device.h | 760 ++ > drivers/gpu/drm/imagination/pvr_device_info.c | 223 + > drivers/gpu/drm/imagination/pvr_device_info.h | 133 + > drivers/gpu/drm/imagination/pvr_drv.c | 1634 +++++ > drivers/gpu/drm/imagination/pvr_drv.h | 89 + > drivers/gpu/drm/imagination/pvr_dump.c | 353 + > drivers/gpu/drm/imagination/pvr_dump.h | 17 + > drivers/gpu/drm/imagination/pvr_free_list.c | 559 ++ > drivers/gpu/drm/imagination/pvr_free_list.h | 185 + > drivers/gpu/drm/imagination/pvr_fw.c | 1107 +++ > drivers/gpu/drm/imagination/pvr_fw.h | 345 + > drivers/gpu/drm/imagination/pvr_fw_info.h | 115 + > drivers/gpu/drm/imagination/pvr_fw_meta.c | 598 ++ > drivers/gpu/drm/imagination/pvr_fw_meta.h | 14 + > drivers/gpu/drm/imagination/pvr_fw_mips.c | 276 + > drivers/gpu/drm/imagination/pvr_fw_mips.h | 38 + > .../gpu/drm/imagination/pvr_fw_startstop.c | 279 + > .../gpu/drm/imagination/pvr_fw_startstop.h | 13 + > drivers/gpu/drm/imagination/pvr_fw_trace.c | 505 ++ > drivers/gpu/drm/imagination/pvr_fw_trace.h | 78 + > drivers/gpu/drm/imagination/pvr_gem.c | 1122 +++ > drivers/gpu/drm/imagination/pvr_gem.h | 386 + > drivers/gpu/drm/imagination/pvr_hwrt.c | 551 ++ > drivers/gpu/drm/imagination/pvr_hwrt.h | 163 + > drivers/gpu/drm/imagination/pvr_job.c | 1096 +++ > drivers/gpu/drm/imagination/pvr_job.h | 116 + > drivers/gpu/drm/imagination/pvr_params.c | 147 + > drivers/gpu/drm/imagination/pvr_params.h | 72 + > drivers/gpu/drm/imagination/pvr_power.c | 196 + > drivers/gpu/drm/imagination/pvr_power.h | 37 + > .../gpu/drm/imagination/pvr_rogue_cr_defs.h | 6193 +++++++++++++++++ > .../imagination/pvr_rogue_cr_defs_client.h | 160 + > drivers/gpu/drm/imagination/pvr_rogue_defs.h | 179 + > drivers/gpu/drm/imagination/pvr_rogue_fwif.h | 2271 ++++++ > .../drm/imagination/pvr_rogue_fwif_check.h | 491 ++ > .../drm/imagination/pvr_rogue_fwif_client.h | 369 + > .../imagination/pvr_rogue_fwif_client_check.h | 133 + > .../drm/imagination/pvr_rogue_fwif_common.h | 60 + > .../pvr_rogue_fwif_resetframework.h | 29 + > .../gpu/drm/imagination/pvr_rogue_fwif_sf.h | 890 +++ > .../drm/imagination/pvr_rogue_fwif_shared.h | 258 + > .../imagination/pvr_rogue_fwif_shared_check.h | 107 + > .../drm/imagination/pvr_rogue_fwif_stream.h | 78 + > .../drm/imagination/pvr_rogue_heap_config.h | 113 + > drivers/gpu/drm/imagination/pvr_rogue_meta.h | 356 + > drivers/gpu/drm/imagination/pvr_rogue_mips.h | 335 + > .../drm/imagination/pvr_rogue_mips_check.h | 56 + > .../gpu/drm/imagination/pvr_rogue_mmu_defs.h | 136 + > drivers/gpu/drm/imagination/pvr_stream.c | 321 + > drivers/gpu/drm/imagination/pvr_stream.h | 74 + > drivers/gpu/drm/imagination/pvr_stream_defs.c | 270 + > drivers/gpu/drm/imagination/pvr_stream_defs.h | 14 + > drivers/gpu/drm/imagination/pvr_vendor.h | 77 + > drivers/gpu/drm/imagination/pvr_vm.c | 3811 ++++++++++ > drivers/gpu/drm/imagination/pvr_vm.h | 99 + > drivers/gpu/drm/imagination/pvr_vm_mips.c | 223 + > drivers/gpu/drm/imagination/pvr_vm_mips.h | 22 + > .../gpu/drm/imagination/vendor/pvr_mt8173.c | 121 + > include/uapi/drm/pvr_drm.h | 1502 ++++ > 71 files changed, 33724 insertions(+) > create mode 100644 Documentation/devicetree/bindings/gpu/img,powervr.yaml > create mode 100644 drivers/gpu/drm/imagination/Kconfig > create mode 100644 drivers/gpu/drm/imagination/Makefile > create mode 100644 drivers/gpu/drm/imagination/pvr_ccb.c > create mode 100644 drivers/gpu/drm/imagination/pvr_ccb.h > create mode 100644 drivers/gpu/drm/imagination/pvr_cccb.c > create mode 100644 drivers/gpu/drm/imagination/pvr_cccb.h > create mode 100644 drivers/gpu/drm/imagination/pvr_context.c > create mode 100644 drivers/gpu/drm/imagination/pvr_context.h > create mode 100644 drivers/gpu/drm/imagination/pvr_debugfs.c > create mode 100644 drivers/gpu/drm/imagination/pvr_debugfs.h > create mode 100644 drivers/gpu/drm/imagination/pvr_device.c > create mode 100644 drivers/gpu/drm/imagination/pvr_device.h > create mode 100644 drivers/gpu/drm/imagination/pvr_device_info.c > create mode 100644 drivers/gpu/drm/imagination/pvr_device_info.h > create mode 100644 drivers/gpu/drm/imagination/pvr_drv.c > create mode 100644 drivers/gpu/drm/imagination/pvr_drv.h > create mode 100644 drivers/gpu/drm/imagination/pvr_dump.c > create mode 100644 drivers/gpu/drm/imagination/pvr_dump.h > create mode 100644 drivers/gpu/drm/imagination/pvr_free_list.c > create mode 100644 drivers/gpu/drm/imagination/pvr_free_list.h > create mode 100644 drivers/gpu/drm/imagination/pvr_fw.c > create mode 100644 drivers/gpu/drm/imagination/pvr_fw.h > create mode 100644 drivers/gpu/drm/imagination/pvr_fw_info.h > create mode 100644 drivers/gpu/drm/imagination/pvr_fw_meta.c > create mode 100644 drivers/gpu/drm/imagination/pvr_fw_meta.h > create mode 100644 drivers/gpu/drm/imagination/pvr_fw_mips.c > create mode 100644 drivers/gpu/drm/imagination/pvr_fw_mips.h > create mode 100644 drivers/gpu/drm/imagination/pvr_fw_startstop.c > create mode 100644 drivers/gpu/drm/imagination/pvr_fw_startstop.h > create mode 100644 drivers/gpu/drm/imagination/pvr_fw_trace.c > create mode 100644 drivers/gpu/drm/imagination/pvr_fw_trace.h > create mode 100644 drivers/gpu/drm/imagination/pvr_gem.c > create mode 100644 drivers/gpu/drm/imagination/pvr_gem.h > create mode 100644 drivers/gpu/drm/imagination/pvr_hwrt.c > create mode 100644 drivers/gpu/drm/imagination/pvr_hwrt.h > create mode 100644 drivers/gpu/drm/imagination/pvr_job.c > create mode 100644 drivers/gpu/drm/imagination/pvr_job.h > create mode 100644 drivers/gpu/drm/imagination/pvr_params.c > create mode 100644 drivers/gpu/drm/imagination/pvr_params.h > create mode 100644 drivers/gpu/drm/imagination/pvr_power.c > create mode 100644 drivers/gpu/drm/imagination/pvr_power.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_cr_defs.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_cr_defs_client.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_defs.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_fwif.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_fwif_check.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_fwif_client.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_fwif_client_check.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_fwif_common.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_fwif_resetframework.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_fwif_sf.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_fwif_shared.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_fwif_shared_check.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_fwif_stream.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_heap_config.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_meta.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_mips.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_mips_check.h > create mode 100644 drivers/gpu/drm/imagination/pvr_rogue_mmu_defs.h > create mode 100644 drivers/gpu/drm/imagination/pvr_stream.c > create mode 100644 drivers/gpu/drm/imagination/pvr_stream.h > create mode 100644 drivers/gpu/drm/imagination/pvr_stream_defs.c > create mode 100644 drivers/gpu/drm/imagination/pvr_stream_defs.h > create mode 100644 drivers/gpu/drm/imagination/pvr_vendor.h > create mode 100644 drivers/gpu/drm/imagination/pvr_vm.c > create mode 100644 drivers/gpu/drm/imagination/pvr_vm.h > create mode 100644 drivers/gpu/drm/imagination/pvr_vm_mips.c > create mode 100644 drivers/gpu/drm/imagination/pvr_vm_mips.h > create mode 100644 drivers/gpu/drm/imagination/vendor/pvr_mt8173.c > create mode 100644 include/uapi/drm/pvr_drm.h > > -- > 2.40.0 > -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
