On Thu, May 2, 2024 at 8:53 AM Sharma, Shashank <shashank.sharma@xxxxxxx> wrote:
>
>
> On 02/05/2024 07:23, Sharma, Shashank wrote:
> > Hey Alex,
> >
> > On 01/05/2024 22:39, Alex Deucher wrote:
> >> On Fri, Apr 26, 2024 at 10:07 AM Shashank Sharma
> >> <shashank.sharma@xxxxxxx> wrote:
> >>> From: Alex Deucher <alexander.deucher@xxxxxxx>
> >>>
> >>> This patch intorduces new UAPI/IOCTL for usermode graphics
> >>> queue. The userspace app will fill this structure and request
> >>> the graphics driver to add a graphics work queue for it. The
> >>> output of this UAPI is a queue id.
> >>>
> >>> This UAPI maps the queue into GPU, so the graphics app can start
> >>> submitting work to the queue as soon as the call returns.
> >>>
> >>> V2: Addressed review comments from Alex and Christian
> >>> - Make the doorbell offset's comment clearer
> >>> - Change the output parameter name to queue_id
> >>>
> >>> V3: Integration with doorbell manager
> >>>
> >>> V4:
> >>> - Updated the UAPI doc (Pierre-Eric)
> >>> - Created a Union for engine specific MQDs (Alex)
> >>> - Added Christian's R-B
> >>> V5:
> >>> - Add variables for GDS and CSA in MQD structure (Alex)
> >>> - Make MQD data a ptr-size pair instead of union (Alex)
> >>>
> >>> V9:
> >>> - renamed struct drm_amdgpu_userq_mqd_gfx_v11 to struct
> >>> drm_amdgpu_userq_mqd as its being used for SDMA and
> >>> compute queues as well
> >>>
> >>> Cc: Alex Deucher <alexander.deucher@xxxxxxx>
> >>> Cc: Christian Koenig <christian.koenig@xxxxxxx>
> >>> Reviewed-by: Christian König <christian.koenig@xxxxxxx>
> >>> Signed-off-by: Alex Deucher <alexander.deucher@xxxxxxx>
> >>> Signed-off-by: Shashank Sharma <shashank.sharma@xxxxxxx>
> >>> ---
> >>> include/uapi/drm/amdgpu_drm.h | 110
> >>> ++++++++++++++++++++++++++++++++++
> >>> 1 file changed, 110 insertions(+)
> >>>
> >>> diff --git a/include/uapi/drm/amdgpu_drm.h
> >>> b/include/uapi/drm/amdgpu_drm.h
> >>> index 96e32dafd4f0..22f56a30f7cb 100644
> >>> --- a/include/uapi/drm/amdgpu_drm.h
> >>> +++ b/include/uapi/drm/amdgpu_drm.h
> >>> @@ -54,6 +54,7 @@ extern "C" {
> >>> #define DRM_AMDGPU_VM 0x13
> >>> #define DRM_AMDGPU_FENCE_TO_HANDLE 0x14
> >>> #define DRM_AMDGPU_SCHED 0x15
> >>> +#define DRM_AMDGPU_USERQ 0x16
> >>>
> >>> #define DRM_IOCTL_AMDGPU_GEM_CREATE DRM_IOWR(DRM_COMMAND_BASE +
> >>> DRM_AMDGPU_GEM_CREATE, union drm_amdgpu_gem_create)
> >>> #define DRM_IOCTL_AMDGPU_GEM_MMAP DRM_IOWR(DRM_COMMAND_BASE +
> >>> DRM_AMDGPU_GEM_MMAP, union drm_amdgpu_gem_mmap)
> >>> @@ -71,6 +72,7 @@ extern "C" {
> >>> #define DRM_IOCTL_AMDGPU_VM DRM_IOWR(DRM_COMMAND_BASE +
> >>> DRM_AMDGPU_VM, union drm_amdgpu_vm)
> >>> #define DRM_IOCTL_AMDGPU_FENCE_TO_HANDLE DRM_IOWR(DRM_COMMAND_BASE
> >>> + DRM_AMDGPU_FENCE_TO_HANDLE, union drm_amdgpu_fence_to_handle)
> >>> #define DRM_IOCTL_AMDGPU_SCHED DRM_IOW(DRM_COMMAND_BASE +
> >>> DRM_AMDGPU_SCHED, union drm_amdgpu_sched)
> >>> +#define DRM_IOCTL_AMDGPU_USERQ DRM_IOW(DRM_COMMAND_BASE +
> >>> DRM_AMDGPU_USERQ, union drm_amdgpu_userq)
> >>>
> >>> /**
> >>> * DOC: memory domains
> >>> @@ -317,6 +319,114 @@ union drm_amdgpu_ctx {
> >>> union drm_amdgpu_ctx_out out;
> >>> };
> >>>
> >>> +/* user queue IOCTL */
> >>> +#define AMDGPU_USERQ_OP_CREATE 1
> >>> +#define AMDGPU_USERQ_OP_FREE 2
> >>> +
> >>> +/* Flag to indicate secure buffer related workload, unused for now */
> >>> +#define AMDGPU_USERQ_MQD_FLAGS_SECURE (1 << 0)
> >>> +/* Flag to indicate AQL workload, unused for now */
> >>> +#define AMDGPU_USERQ_MQD_FLAGS_AQL (1 << 1)
> >>> +
> >>> +/*
> >>> + * MQD (memory queue descriptor) is a set of parameters which allow
> >>> + * the GPU to uniquely define and identify a usermode queue. This
> >>> + * structure defines the MQD for GFX-V11 IP ver 0.
> >>> + */
> >>> +struct drm_amdgpu_userq_mqd {
> >> Maybe rename this to drm_amdgpu_gfx_userq_mqd since it's gfx specific.
> >> Then we can add different MQDs for SDMA, compute, etc. as they have
> >> different metadata. E.g., the shadow and CSA are gfx only.
> >
> >
> > Actually this was named drm_amdgpu_userq_mqd_gfx_v11_0 until the last
> > patchset, but then I realized that apart from the objects (gds/shadow
> > va) nothing is gfx specific, its actually required for every userqueue
> > IP which is MES based, so I thought it would be an overkill to create
> > multiple structures for almost the same data. If you feel strong about
> > this, I can change it again.
> >
> > - Shashank
>
>
> Please ignore my last comment, I understand what you are mentioning, and
> I have reformatted the patches accordingly. Now, I am keeping everything
> reqd for MES in one basic struture (drm_amdgpu_userq_in) and creating
> drm_amdgpu_userq_mqd_gfx_v11 for GFX specific things (like CSA, Shadow
> and GDS areas). Now there will be one separate patch which will enabled
> GFX_IP on MES code, just like how we have separate patches for SDMA and
> Compute IP in this series. I will send the V10 patches with this
> reformatting in some time.
Yeah, we just need to make it clear to userspace what buffers are
necessary for which ring type.
Alex
>
> - Shashank
>
> >
> >> Alex
> >>
> >>
> >>> + /**
> >>> + * @queue_va: Virtual address of the GPU memory which holds
> >>> the queue
> >>> + * object. The queue holds the workload packets.
> >>> + */
> >>> + __u64 queue_va;
> >>> + /**
> >>> + * @queue_size: Size of the queue in bytes, this needs to be
> >>> 256-byte
> >>> + * aligned.
> >>> + */
> >>> + __u64 queue_size;
> >>> + /**
> >>> + * @rptr_va : Virtual address of the GPU memory which holds
> >>> the ring RPTR.
> >>> + * This object must be at least 8 byte in size and aligned
> >>> to 8-byte offset.
> >>> + */
> >>> + __u64 rptr_va;
> >>> + /**
> >>> + * @wptr_va : Virtual address of the GPU memory which holds
> >>> the ring WPTR.
> >>> + * This object must be at least 8 byte in size and aligned
> >>> to 8-byte offset.
> >>> + *
> >>> + * Queue, RPTR and WPTR can come from the same object, as
> >>> long as the size
> >>> + * and alignment related requirements are met.
> >>> + */
> >>> + __u64 wptr_va;
> >>> + /**
> >>> + * @shadow_va: Virtual address of the GPU memory to hold the
> >>> shadow buffer.
> >>> + * This must be a from a separate GPU object, and must be at
> >>> least 4-page
> >>> + * sized.
> >>> + */
> >>> + __u64 shadow_va;
> >>> + /**
> >>> + * @gds_va: Virtual address of the GPU memory to hold the
> >>> GDS buffer.
> >>> + * This must be a from a separate GPU object, and must be at
> >>> least 1-page
> >>> + * sized.
> >>> + */
> >>> + __u64 gds_va;
> >>> + /**
> >>> + * @csa_va: Virtual address of the GPU memory to hold the
> >>> CSA buffer.
> >>> + * This must be a from a separate GPU object, and must be at
> >>> least 1-page
> >>> + * sized.
> >>> + */
> >>> + __u64 csa_va;
> >>> +};
> >>> +
> >>> +struct drm_amdgpu_userq_in {
> >>> + /** AMDGPU_USERQ_OP_* */
> >>> + __u32 op;
> >>> + /** Queue handle for USERQ_OP_FREE */
> >>> + __u32 queue_id;
> >>> + /** the target GPU engine to execute workload
> >>> (AMDGPU_HW_IP_*) */
> >>> + __u32 ip_type;
> >>> + /**
> >>> + * @flags: flags to indicate special function for queue like
> >>> secure
> >>> + * buffer (TMZ). Unused for now.
> >>> + */
> >>> + __u32 flags;
> >>> + /**
> >>> + * @doorbell_handle: the handle of doorbell GEM object
> >>> + * associated to this client.
> >>> + */
> >>> + __u32 doorbell_handle;
> >>> + /**
> >>> + * @doorbell_offset: 32-bit offset of the doorbell in the
> >>> doorbell bo.
> >>> + * Kernel will generate absolute doorbell offset using
> >>> doorbell_handle
> >>> + * and doorbell_offset in the doorbell bo.
> >>> + */
> >>> + __u32 doorbell_offset;
> >>> + /**
> >>> + * @mqd: Queue descriptor for USERQ_OP_CREATE
> >>> + * MQD data can be of different size for different GPU
> >>> IP/engine and
> >>> + * their respective versions/revisions, so this points to a
> >>> __u64 *
> >>> + * which holds MQD of this usermode queue.
> >>> + */
> >>> + __u64 mqd;
> >>> + /**
> >>> + * @size: size of MQD data in bytes, it must match the MQD
> >>> structure
> >>> + * size of the respective engine/revision defined in UAPI
> >>> for ex, for
> >>> + * gfx_v11 workloads, size =
> >>> sizeof(drm_amdgpu_userq_mqd_gfx_v11).
> >>> + */
> >>> + __u64 mqd_size;
> >>> +};
> >>> +
> >>> +struct drm_amdgpu_userq_out {
> >>> + /** Queue handle */
> >>> + __u32 queue_id;
> >>> + /** Flags */
> >>> + __u32 flags;
> >>> +};
> >>> +
> >>> +union drm_amdgpu_userq {
> >>> + struct drm_amdgpu_userq_in in;
> >>> + struct drm_amdgpu_userq_out out;
> >>> +};
> >>> +
> >>> /* vm ioctl */
> >>> #define AMDGPU_VM_OP_RESERVE_VMID 1
> >>> #define AMDGPU_VM_OP_UNRESERVE_VMID 2
> >>> --
> >>> 2.43.2
> >>>
