On 6/22/23 21:13, Benjamin Gaignard wrote:
CAUTION: Email originated externally, do not click links or open attachments unless you recognize the sender and know the content is safe. Unlike when resolution change on keyframes, dynamic resolution change on inter frames doesn't allow to do a stream off/on sequence because it is need to keep all previous references alive to decode inter frames. This constraint have two main problems: - more memory consumption. - more buffers in use. To solve these issue this series introduce DELETE_BUF ioctl and remove the 32 buffers limit per queue.
I know the VIDIOC_CREATE_BUFS allows creating a buffer with a different size than the driver suggests in G_FMT.
But the vb2_ops->queue_setup() could check whether the sizeimages meet its minimal requirement with the current format.
This enables a problem that the driver need to check the buffer size before they make a hardware use a buffer from the rdy_queue.
Thinking of such case, we know a AV1 sequence(VP9 or VP8 didn't have a sequence header) would need a much large buffer for the alternative reference frame.
Then create one special buffer for the altref, the driver need a hardware to pick it from the rdy_queue first or it would be a waste to use it as a regular frame buffer.
Also missing such step would not solve the memory allocation problem.
VP9 conformance tests using fluster give a score of 210/305. The 25 resize inter tests (vp90-2-21-resize_inter_* files) are ok but require to use postprocessor. Kernel branch is available here: https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.collabora.com_benjamin.gaignard_for-2Dupstream_-2D_commits_remove-5Fvb2-5Fqueue-5Flimit-5Fv3&d=DwIDAg&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=DCpeuc2fAyJ_XUCYsydYOB5ynn0uW4JsFKVbEiXj-6AhZ5d2vm3GkOClPl8cfN9U&s=8whob9PKPu98WlyK6J9DcmFFiDPbwI3ws-nLfWR0oTE&e= GStreamer branch to use DELETE_BUF ioctl and testing dynamic resolution change is here: https://urldefense.proofpoint.com/v2/url?u=https-3A__gitlab.freedesktop.org_benjamin.gaignard1_gstreamer_-2D_commits_VP9-5Fdrc&d=DwIDAg&c=7dfBJ8cXbWjhc0BhImu8wVIoUFmBzj1s88r8EGyM0UY&r=P4xb2_7biqBxD4LGGPrSV6j-jf3C3xlR7PXU-mLTeZE&m=DCpeuc2fAyJ_XUCYsydYOB5ynn0uW4JsFKVbEiXj-6AhZ5d2vm3GkOClPl8cfN9U&s=SEexoIeuXbraR1zvtSkz0MQFGyZSeKQ7Pt6mJoNrS0A&e= changes in version 3: - Use Xarray API to store allocated video buffers. - No module parameter to limit the number of buffer per queue. - Use Xarray inside Verisilicon driver to store postprocessor buffers and remove VB2_MAX_FRAME limit. - Allow Versilicon driver to change of resolution while streaming - Various fixes the Verisilicon VP9 code to improve fluster score. changes in version 2: - Use a dynamic array and not a list to keep trace of allocated buffers. Not use IDR interface because it is marked as deprecated in kernel documentation. - Add a module parameter to limit the number of buffer per queue. - Add DELETE_BUF ioctl and m2m helpers. Benjamin Gaignard (11): media: videobuf2: Access vb2_queue bufs array through helper functions media: videobuf2: Use Xarray instead of static buffers array media: videobuf2: Remove VB2_MAX_FRAME limit on buffer storage media: videobuf2: Stop define VB2_MAX_FRAME as global media: verisilicon: Refactor postprocessor to store more buffers media: verisilicon: Store chroma and motion vectors offset media: verisilicon: vp9: Use destination buffer height to compute chroma offset media: verisilicon: postproc: Fix down scale test media: verisilicon: vp9: Allow to change resolution while streaming media: v4l2: Add DELETE_BUF ioctl media: v4l2: Add mem2mem helpers for DELETE_BUF ioctl .../userspace-api/media/v4l/user-func.rst | 1 + .../media/v4l/vidioc-delete-buf.rst | 51 ++++ .../media/common/videobuf2/videobuf2-core.c | 275 ++++++++++++++---- .../media/common/videobuf2/videobuf2-v4l2.c | 34 ++- drivers/media/platform/amphion/vdec.c | 1 + drivers/media/platform/amphion/vpu_dbg.c | 22 +- .../platform/mediatek/jpeg/mtk_jpeg_core.c | 6 +- .../vcodec/vdec/vdec_vp9_req_lat_if.c | 4 +- drivers/media/platform/qcom/venus/hfi.h | 2 + drivers/media/platform/st/sti/hva/hva-v4l2.c | 4 + drivers/media/platform/verisilicon/hantro.h | 8 +- .../platform/verisilicon/hantro_g2_vp9_dec.c | 10 +- .../media/platform/verisilicon/hantro_hw.h | 4 +- .../platform/verisilicon/hantro_postproc.c | 114 +++++--- .../media/platform/verisilicon/hantro_v4l2.c | 37 +-- drivers/media/test-drivers/vim2m.c | 1 + drivers/media/test-drivers/visl/visl-dec.c | 28 +- drivers/media/v4l2-core/v4l2-dev.c | 1 + drivers/media/v4l2-core/v4l2-ioctl.c | 10 + drivers/media/v4l2-core/v4l2-mem2mem.c | 20 ++ .../staging/media/atomisp/pci/atomisp_ioctl.c | 2 +- drivers/staging/media/ipu3/ipu3-v4l2.c | 2 + include/media/v4l2-ioctl.h | 4 + include/media/v4l2-mem2mem.h | 12 + include/media/videobuf2-core.h | 16 +- include/media/videobuf2-v4l2.h | 15 +- include/uapi/linux/videodev2.h | 2 + 27 files changed, 523 insertions(+), 163 deletions(-) create mode 100644 Documentation/userspace-api/media/v4l/vidioc-delete-buf.rst -- 2.39.2
-- Hsia-Jun(Randy) Li
