On Wed, Apr 6, 2022 at 7:41 AM Benjamin Gaignard <benjamin.gaignard@xxxxxxxxxxxxx> wrote: > > > Le 06/04/2022 à 14:28, Adam Ford a écrit : > > On Wed, Apr 6, 2022 at 1:56 AM Benjamin Gaignard > > <benjamin.gaignard@xxxxxxxxxxxxx> wrote: > >> > >> Le 05/04/2022 à 23:27, Adam Ford a écrit : > >>> On Mon, Apr 4, 2022 at 10:56 AM Benjamin Gaignard > >>> <benjamin.gaignard@xxxxxxxxxxxxx> wrote: > >>>> Le 02/04/2022 à 18:59, Adam Ford a écrit : > >>>>> On Sat, Apr 2, 2022 at 11:22 AM Adam Ford <aford173@xxxxxxxxx> wrote: > >>>>>> On Fri, Apr 1, 2022 at 8:18 AM Benjamin Gaignard > >>>>>> <benjamin.gaignard@xxxxxxxxxxxxx> wrote: > >>>>>>> Le 31/03/2022 à 08:53, Benjamin Gaignard a écrit : > >>>>>>>> Le 30/03/2022 à 20:52, Adam Ford a écrit : > >>>>>>>>> On Wed, Mar 30, 2022 at 2:53 AM Benjamin Gaignard > >>>>>>>>> <benjamin.gaignard@xxxxxxxxxxxxx> wrote: > >>>>>>>>>> Le 28/02/2022 à 15:08, Benjamin Gaignard a écrit : > >>>>>>>>>>> This series aims to make HEVC uapi stable and usable for hardware > >>>>>>>>>>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro) > >>>>>>>>>>> and 2 out of the tree drivers (rkvdec and RPI). > >>>>>>>>>>> > >>>>>>>>>>> After the remarks done on version 2, I have completely reworked to > >>>>>>>>>>> patches > >>>>>>>>>>> split so changelogs are meaningless. I have also drop "RFC" from the > >>>>>>>>>>> titles. > >>>>>>>>>>> > >>>>>>>>>>> Version 4: > >>>>>>>>>>> - Add num_entry_point_offsets field in struct > >>>>>>>>>>> v4l2_ctrl_hevc_slice_params > >>>>>>>>>>> - Fix V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS name > >>>>>>>>>>> - Initialize control V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS > >>>>>>>>>>> - Fix space/tab issue in kernel-doc > >>>>>>>>>>> - Add patch to change data_bit_offset definition > >>>>>>>>>>> - Fix hantro-media SPDX license > >>>>>>>>>>> - put controls under stateless section in v4l2-ctrls-defs.c > >>>>>>>>>>> > >>>>>>>>>>> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec. > >>>>>>>>>> Dear reviewers, > >>>>>>>>>> > >>>>>>>>>> This series is waiting for your feedback, > >>>>>>>>> I tried several times with the suggested repos for both the kernel and > >>>>>>>>> g-streamer without success getting Fluster to pass any tests on the > >>>>>>>>> imx8mq. I can try again but I likely won't get to it until this > >>>>>>>>> weekend. If I can get it working, I'll test both the 8mq and 8mm. > >>>>>>>> Thanks a lot for that. > >>>>>>>> > >>>>>>>> Benjamin > >>>>>>> Adam, > >>>>>>> > >>>>>>> You may need to check if h265parse and v4l2slh265dec are available on your board. > >>>>>> I ran gst-inspect to see what showed up with 265 in the name. > >>>>>> > >>>>>> # gst-inspect-1.0 |grep 265 > >>>>>> libav: avdec_h265: libav HEVC (High Efficiency Video Coding) decoder > >>>>>> rtp: rtph265depay: RTP H265 depayloader > >>>>>> rtp: rtph265pay: RTP H265 payloader > >>>>>> typefindfunctions: video/x-h265: h265, x265, 265 > >>>>>> v4l2codecs: v4l2slh265dec: V4L2 Stateless H.265 Video Decoder > >>>>>> videoparsersbad: h265parse: H.265 parser > >>>>>> > >>>>>> It appears I have both h265parse and v4l2slh265dec. > >>>>>> > >>>>>>> fluster check if v4l2slh265dec is working fine with this command line: > >>>>>>> > >>>>>>> gst-launch-1.0 appsrc num-buffers=0 ! h265parse ! v4l2slh265dec ! fakesink > >>>>>>> > >>>>>>> so if one of them is missing it won't work. > >>>>>> gst-launch-1.0 appsrc num-buffers=0 ! h265parse ! v4l2slh265dec ! fakesink > >>>>>> Setting pipeline to PAUSED ... > >>>>>> 0:00:00.098389938 526 0xaaaaf9d86ac0 ERROR v4l2codecs-decoder > >>>>>> gstv4l2decoder.c:725:gst_v4l2_decoder_get_controls:<v4l2decoder2> > >>>>>> VIDIOC_G_EXT_CTRLS failed: Invalid argument > >>>>>> ERROR: from element > >>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Driver did not > >>>>>> report framing and start code method. > >>>>>> Additional debug info: > >>>>>> ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codech265dec.c(155): > >>>>>> gst_v4l2_codec_h265_dec_open (): > >>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: > >>>>>> gst_v4l2_decoder_get_controls() failed: Invalid argument > >>>>>> ERROR: pipeline doesn't want to preroll. > >>>>>> ERROR: from element > >>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Could not > >>>>>> initialize supporting library. > >>>>>> Additional debug info: > >>>>>> ../subprojects/gst-plugins-base/gst-libs/gst/video/gstvideodecoder.c(2909): > >>>>>> gst_video_decoder_change_state (): > >>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: > >>>>>> Failed to open decoder > >>>>>> ERROR: pipeline doesn't want to preroll. > >>>>>> Failed to set pipeline to PAUSED. > >>>>>> Setting pipeline to NULL ... > >>>>>> Freeing pipeline ... > >>>>>> > >>>>>> Does this mean I have a wrong version of the kernel and/or incomplete patches? > >>>>> I double checked the branches. > >>>>> > >>>>> Kernel: > >>>>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream.git > >>>>> branch: origin/HEVC_UAPI_V4 > >>>>> > >>>>> Gstreamer: > >>>>> https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer.git > >>>>> branch: origin/benjamin.gaignard1/gstreamer-HEVC_aligned_with_kernel_5.15 > >>>>> > >>>>> > >>>>> I am still not able to run h.265/HEVC tests. > >>>> Hello Adam, > >>>> > >>>> I have updated the following branches with the versions I have used today: > >>>> > >>>> Kernel: > >>>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream.git > >>>> branch: origin/HEVC_UAPI_V5 only one change in documentation vs version 4 but rebased in v5.18-rc1 > >>>> > >>>> Gstreamer: > >>>> https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer.git > >>>> branch: origin/benjamin.gaignard1/gstreamer-HEVC_aligned_with_kernel_5.15 updated on the latest GST main branch > >>>> > >>>> I hope this will work fine this time. > >>> I wish I had better news for you: > >>> > >>> dmesg shows the hantro driver is being loaded: > >>> > >>> [ 38.612243] hantro-vpu 38300000.video-codec: registered > >>> nxp,imx8mq-vpu-g1-dec as /dev/video0 > >>> [ 38.612618] hantro-vpu 38310000.video-codec: registered > >>> nxp,imx8mq-vpu-g2-dec as /dev/video1 > >>> > >>> # gst-inspect-1.0 |grep 265 > >>> libav: avdec_h265: libav HEVC (High Efficiency Video Coding) decoder > >>> rtp: rtph265depay: RTP H265 depayloader > >>> rtp: rtph265pay: RTP H265 payloader > >>> typefindfunctions: video/x-h265: h265, x265, 265 > >>> v4l2codecs: v4l2slh265dec: V4L2 Stateless H.265 Video Decoder > >>> videoparsersbad: h265parse: H.265 parser > >>> > >>> Fluster reports: > >>> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for > >>> GStreamer 1.0... ❌ > >> Still the same error about non supported control ? > > # gst-lagst-launch-1.0 appsrc num-buffers=0 ! h265parse ! > > v4l2slh265dec ! fakesink > > Setting pipeline to PAUSED ... > > 0:00:01.704385508 420 0xaaaaec0a76c0 ERROR v4l2codecs-decoder > > gstv4l2decoder.c:725:gst_v4l2_decoder_get_controls:<v4l2decoder2> > > VIDIOC_G_EXT_CTRLS failed: Invalid argument > > ERROR: from element > > /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Driver did not > > report framing and start code method. > > Additional debug info: > > ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codech265dec.c(155): > > gst_v4l2_codec_h265_dec_open (): > > /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: > > gst_v4l2_decoder_get_controls() failed: Invalid argument > > ERROR: pipeline doesn't want to preroll. > > ERROR: from element > > /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Could not > > initialize supporting library. > > Additional debug info: > > ../subprojects/gst-plugins-base/gst-libs/gst/video/gstvideodecoder.c(2909): > > gst_video_decoder_change_state (): > > /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: > > Failed to open decoder > > ERROR: pipeline doesn't want to preroll. > > Failed to set pipeline to PAUSED. > > Setting pipeline to NULL ... > > Freeing pipeline ... > > What is the result of v4l2-ctl ? > Here I have: > v4l2-ctl --all -d 1 > Driver Info: > Driver name : hantro-vpu > Card type : nxp,imx8mq-vpu-g2-dec > Bus info : platform: hantro-vpu > Driver version : 5.18.0 > Capabilities : 0x84204000 > Video Memory-to-Memory Multiplanar > Streaming > Extended Pix Format > Device Capabilities > Device Caps : 0x04204000 > Video Memory-to-Memory Multiplanar > Streaming > Extended Pix Format > Media Driver Info: > Driver name : hantro-vpu > Model : hantro-vpu > Serial : > Bus info : platform: hantro-vpu > Media version : 5.18.0 > Hardware revision: 0x00000000 (0) > Driver version : 5.18.0 > Interface Info: > ID : 0x0300000c > Type : V4L Video > Entity Info: > ID : 0x00000001 (1) > Name : nxp,imx8mq-vpu-g2-dec-source > Function : V4L2 I/O > Pad 0x01000002 : 0: Source > Link 0x02000008: to remote pad 0x1000004 of entity 'nxp,imx8mq-vpu-g2-dec-proc': Data, Enabled, Immutable > Priority: 2 > Format Video Capture Multiplanar: > Width/Height : 48/48 > Pixel Format : 'VT12' (Y/CbCr 4:2:0 (4x4 Linear)) > Field : None > Number of planes : 1 > Flags : > Colorspace : JPEG > Transfer Function : Default > YCbCr/HSV Encoding: Default > Quantization : Default > Plane 0 : > Bytes per Line : 48 > Size Image : 3600 > Format Video Output Multiplanar: > Width/Height : 48/48 > Pixel Format : 'S265' (HEVC Parsed Slice Data) > Field : None > Number of planes : 1 > Flags : > Colorspace : JPEG > Transfer Function : Default > YCbCr/HSV Encoding: Default > Quantization : Default > Plane 0 : > Bytes per Line : 0 > Size Image : 4608 > > User Controls > > hantro_hevc_slice_header_skip_b 0x00981a80 (int) : min=0 max=256 step=1 default=0 value=0 > > Codec Controls > > hevc_profile 0x00990b67 (menu) : min=0 max=2 default=0 value=0 > 0: Main > 1: Main Still Picture > 2: Main 10 > hevc_level 0x00990b68 (menu) : min=0 max=8 default=0 value=0 > 0: 1 > 1: 2 > 2: 2.1 > 3: 3 > 4: 3.1 > 5: 4 > 6: 4.1 > 7: 5 > 8: 5.1 > > Stateless Codec Controls > > vp9_frame_decode_parameters 0x00a40a2c (unknown): type=261 flags=has-payload > vp9_probabilities_updates 0x00a40a2d (unknown): type=260 flags=has-payload > hevc_sequence_parameter_set 0x00a40a90 (unknown): type=270 flags=has-payload > hevc_picture_parameter_set 0x00a40a91 (unknown): type=271 flags=has-payload > hevc_scaling_matrix 0x00a40a93 (unknown): type=273 flags=has-payload > hevc_decode_parameters 0x00a40a94 (unknown): type=274 flags=has-payload > hevc_decode_mode 0x00a40a95 (menu) : min=1 max=1 default=1 value=1 > 1: Frame-Based > hevc_start_code 0x00a40a96 (menu) : min=1 max=1 default=1 value=1 > 1: Annex B Start Code > Mine looks like: # v4l2-ctl --all -d 1 Driver Info: Driver name : hantro-vpu Card type : nxp,imx8mq-vpu-g2-dec Bus info : platform: hantro-vpu Driver version : 5.18.0 Capabilities : 0x84204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Media Driver Info: Driver name : hantro-vpu Model : hantro-vpu Serial : Bus info : platform: hantro-vpu Media version : 5.18.0 Hardware revision: 0x00000000 (0) Driver version : 5.18.0 Interface Info: ID : 0x0300000c Type : V4L Video Entity Info: ID : 0x00000001 (1) Name : nxp,imx8mq-vpu-g2-dec-source Function : V4L2 I/O Pad 0x01000002 : 0: Source Link 0x02000008: to remote pad 0x1000004 of entity 'nxp,imx8mq-vpu-g2-dec-proc': Data, Enabled, Immutable Priority: 2 Format Video Capture Multiplanar: Width/Height : 48/48 Pixel Format : 'VT12' (Y/CbCr 4:2:0 (4x4 Linear)) Field : None Number of planes : 1 Flags : Colorspace : JPEG Transfer Function : Default YCbCr/HSV Encoding: Default Quantization : Default Plane 0 : Bytes per Line : 48 Size Image : 3600 Format Video Output Multiplanar: Width/Height : 48/48 Pixel Format : 'S265' (HEVC Parsed Slice Data) Field : None Number of planes : 1 Flags : Colorspace : JPEG Transfer Function : Default YCbCr/HSV Encoding: Default Quantization : Default Plane 0 : Bytes per Line : 0 Size Image : 4608 User Controls hantro_hevc_slice_header_skip_b 0x00981a80 (int) : min=0 max=256 step=1 default=0 value=0 Codec Controls hevc_profile 0x00990b67 (menu) : min=0 max=2 default=0 value=0 0: Main 1: Main Still Picture 2: Main 10 hevc_level 0x00990b68 (menu) : min=0 max=8 default=0 value=0 0: 1 1: 2 2: 2.1 3: 3 4: 3.1 5: 4 6: 4.1 7: 5 8: 5.1 Stateless Codec Controls vp9_frame_decode_parameters 0x00a40a2c (unknown): type=261 flags=has-payload vp9_probabilities_updates 0x00a40a2d (unknown): type=260 flags=has-payload hevc_sequence_parameter_set 0x00a40a90 (unknown): type=270 flags=has-payload hevc_picture_parameter_set 0x00a40a91 (unknown): type=271 flags=has-payload hevc_scaling_matrix 0x00a40a93 (unknown): type=273 flags=has-payload hevc_decode_parameters 0x00a40a94 (unknown): type=274 flags=has-payload hevc_decode_mode 0x00a40a95 (menu) : min=1 max=1 default=1 value=1 1: Frame-Based hevc_start_code 0x00a40a96 (menu) : min=1 max=1 default=1 value=1 1: Annex B Start Code > > > >> Benjamin > >> > >>> adam > >>>> Benjamin > >>>> > >>>>> adam > >>>>>> adam > >>>>>>> Regards, > >>>>>>> Benjamin > >>>>>>> > >>>>>>>>> adam > >>>>>>>>>> Thanks, > >>>>>>>>>> Benjamin > >>>>>>>>>> > >>>>>>>>>>> Benjamin > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Benjamin Gaignard (12): > >>>>>>>>>>> media: uapi: HEVC: Add missing fields in HEVC controls > >>>>>>>>>>> media: uapi: HEVC: Rename HEVC stateless controls with STATELESS > >>>>>>>>>>> prefix > >>>>>>>>>>> media: uapi: HEVC: Add document uAPI structure > >>>>>>>>>>> media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS > >>>>>>>>>>> as a > >>>>>>>>>>> dynamic array > >>>>>>>>>>> media: uapi: Move parsed HEVC pixel format out of staging > >>>>>>>>>>> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS > >>>>>>>>>>> control > >>>>>>>>>>> media: uapi: Move the HEVC stateless control type out of staging > >>>>>>>>>>> media: controls: Log HEVC stateless control in .std_log > >>>>>>>>>>> media: uapi: Create a dedicated header for Hantro control > >>>>>>>>>>> media: uapi: HEVC: fix padding in v4l2 control structures > >>>>>>>>>>> media: uapi: Change data_bit_offset definition > >>>>>>>>>>> media: uapi: move HEVC stateless controls out of staging > >>>>>>>>>>> > >>>>>>>>>>> Hans Verkuil (3): > >>>>>>>>>>> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY > >>>>>>>>>>> v4l2-ctrls: add support for dynamically allocated arrays. > >>>>>>>>>>> vivid: add dynamic array test control > >>>>>>>>>>> > >>>>>>>>>>> .../userspace-api/media/drivers/hantro.rst | 5 - > >>>>>>>>>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 833 > >>>>>>>>>>> ++++++++++++++++++ > >>>>>>>>>>> .../media/v4l/ext-ctrls-codec.rst | 780 > >>>>>>>>>>> ---------------- > >>>>>>>>>>> .../media/v4l/pixfmt-compressed.rst | 7 +- > >>>>>>>>>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 20 + > >>>>>>>>>>> .../media/v4l/vidioc-queryctrl.rst | 8 + > >>>>>>>>>>> .../media/videodev2.h.rst.exceptions | 5 + > >>>>>>>>>>> .../media/test-drivers/vivid/vivid-ctrls.c | 15 + > >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++- > >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++- > >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 37 +- > >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +- > >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +- > >>>>>>>>>>> drivers/staging/media/hantro/hantro_drv.c | 27 +- > >>>>>>>>>>> drivers/staging/media/hantro/hantro_hevc.c | 8 +- > >>>>>>>>>>> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +- > >>>>>>>>>>> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +- > >>>>>>>>>>> .../staging/media/sunxi/cedrus/cedrus_h265.c | 2 +- > >>>>>>>>>>> include/media/hevc-ctrls.h | 250 ------ > >>>>>>>>>>> include/media/v4l2-ctrls.h | 48 +- > >>>>>>>>>>> include/uapi/linux/hantro-media.h | 19 + > >>>>>>>>>>> include/uapi/linux/v4l2-controls.h | 439 +++++++++ > >>>>>>>>>>> include/uapi/linux/videodev2.h | 13 + > >>>>>>>>>>> 23 files changed, 1697 insertions(+), 1170 deletions(-) > >>>>>>>>>>> delete mode 100644 include/media/hevc-ctrls.h > >>>>>>>>>>> create mode 100644 include/uapi/linux/hantro-media.h > >>>>>>>>>>>
