On Wed, Apr 6, 2022 at 7:50 AM Benjamin Gaignard <benjamin.gaignard@xxxxxxxxxxxxx> wrote: > > > Le 06/04/2022 à 14:46, Adam Ford a écrit : > > On Wed, Apr 6, 2022 at 7:41 AM Benjamin Gaignard > > <benjamin.gaignard@xxxxxxxxxxxxx> wrote: > >> > >> Le 06/04/2022 à 14:28, Adam Ford a écrit : > >>> On Wed, Apr 6, 2022 at 1:56 AM Benjamin Gaignard > >>> <benjamin.gaignard@xxxxxxxxxxxxx> wrote: > >>>> Le 05/04/2022 à 23:27, Adam Ford a écrit : > >>>>> On Mon, Apr 4, 2022 at 10:56 AM Benjamin Gaignard > >>>>> <benjamin.gaignard@xxxxxxxxxxxxx> wrote: > >>>>>> Le 02/04/2022 à 18:59, Adam Ford a écrit : > >>>>>>> On Sat, Apr 2, 2022 at 11:22 AM Adam Ford <aford173@xxxxxxxxx> wrote: > >>>>>>>> On Fri, Apr 1, 2022 at 8:18 AM Benjamin Gaignard > >>>>>>>> <benjamin.gaignard@xxxxxxxxxxxxx> wrote: > >>>>>>>>> Le 31/03/2022 à 08:53, Benjamin Gaignard a écrit : > >>>>>>>>>> Le 30/03/2022 à 20:52, Adam Ford a écrit : > >>>>>>>>>>> On Wed, Mar 30, 2022 at 2:53 AM Benjamin Gaignard > >>>>>>>>>>> <benjamin.gaignard@xxxxxxxxxxxxx> wrote: > >>>>>>>>>>>> Le 28/02/2022 à 15:08, Benjamin Gaignard a écrit : > >>>>>>>>>>>>> This series aims to make HEVC uapi stable and usable for hardware > >>>>>>>>>>>>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro) > >>>>>>>>>>>>> and 2 out of the tree drivers (rkvdec and RPI). > >>>>>>>>>>>>> > >>>>>>>>>>>>> After the remarks done on version 2, I have completely reworked to > >>>>>>>>>>>>> patches > >>>>>>>>>>>>> split so changelogs are meaningless. I have also drop "RFC" from the > >>>>>>>>>>>>> titles. > >>>>>>>>>>>>> > >>>>>>>>>>>>> Version 4: > >>>>>>>>>>>>> - Add num_entry_point_offsets field in struct > >>>>>>>>>>>>> v4l2_ctrl_hevc_slice_params > >>>>>>>>>>>>> - Fix V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS name > >>>>>>>>>>>>> - Initialize control V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS > >>>>>>>>>>>>> - Fix space/tab issue in kernel-doc > >>>>>>>>>>>>> - Add patch to change data_bit_offset definition > >>>>>>>>>>>>> - Fix hantro-media SPDX license > >>>>>>>>>>>>> - put controls under stateless section in v4l2-ctrls-defs.c > >>>>>>>>>>>>> > >>>>>>>>>>>>> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec. > >>>>>>>>>>>> Dear reviewers, > >>>>>>>>>>>> > >>>>>>>>>>>> This series is waiting for your feedback, > >>>>>>>>>>> I tried several times with the suggested repos for both the kernel and > >>>>>>>>>>> g-streamer without success getting Fluster to pass any tests on the > >>>>>>>>>>> imx8mq. I can try again but I likely won't get to it until this > >>>>>>>>>>> weekend. If I can get it working, I'll test both the 8mq and 8mm. > >>>>>>>>>> Thanks a lot for that. > >>>>>>>>>> > >>>>>>>>>> Benjamin > >>>>>>>>> Adam, > >>>>>>>>> > >>>>>>>>> You may need to check if h265parse and v4l2slh265dec are available on your board. > >>>>>>>> I ran gst-inspect to see what showed up with 265 in the name. > >>>>>>>> > >>>>>>>> # gst-inspect-1.0 |grep 265 > >>>>>>>> libav: avdec_h265: libav HEVC (High Efficiency Video Coding) decoder > >>>>>>>> rtp: rtph265depay: RTP H265 depayloader > >>>>>>>> rtp: rtph265pay: RTP H265 payloader > >>>>>>>> typefindfunctions: video/x-h265: h265, x265, 265 > >>>>>>>> v4l2codecs: v4l2slh265dec: V4L2 Stateless H.265 Video Decoder > >>>>>>>> videoparsersbad: h265parse: H.265 parser > >>>>>>>> > >>>>>>>> It appears I have both h265parse and v4l2slh265dec. > >>>>>>>> > >>>>>>>>> fluster check if v4l2slh265dec is working fine with this command line: > >>>>>>>>> > >>>>>>>>> gst-launch-1.0 appsrc num-buffers=0 ! h265parse ! v4l2slh265dec ! fakesink > >>>>>>>>> > >>>>>>>>> so if one of them is missing it won't work. > >>>>>>>> gst-launch-1.0 appsrc num-buffers=0 ! h265parse ! v4l2slh265dec ! fakesink > >>>>>>>> Setting pipeline to PAUSED ... > >>>>>>>> 0:00:00.098389938 526 0xaaaaf9d86ac0 ERROR v4l2codecs-decoder > >>>>>>>> gstv4l2decoder.c:725:gst_v4l2_decoder_get_controls:<v4l2decoder2> > >>>>>>>> VIDIOC_G_EXT_CTRLS failed: Invalid argument > >>>>>>>> ERROR: from element > >>>>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Driver did not > >>>>>>>> report framing and start code method. > >>>>>>>> Additional debug info: > >>>>>>>> ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codech265dec.c(155): > >>>>>>>> gst_v4l2_codec_h265_dec_open (): > >>>>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: > >>>>>>>> gst_v4l2_decoder_get_controls() failed: Invalid argument > >>>>>>>> ERROR: pipeline doesn't want to preroll. > >>>>>>>> ERROR: from element > >>>>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Could not > >>>>>>>> initialize supporting library. > >>>>>>>> Additional debug info: > >>>>>>>> ../subprojects/gst-plugins-base/gst-libs/gst/video/gstvideodecoder.c(2909): > >>>>>>>> gst_video_decoder_change_state (): > >>>>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: > >>>>>>>> Failed to open decoder > >>>>>>>> ERROR: pipeline doesn't want to preroll. > >>>>>>>> Failed to set pipeline to PAUSED. > >>>>>>>> Setting pipeline to NULL ... > >>>>>>>> Freeing pipeline ... > >>>>>>>> > >>>>>>>> Does this mean I have a wrong version of the kernel and/or incomplete patches? > >>>>>>> I double checked the branches. > >>>>>>> > >>>>>>> Kernel: > >>>>>>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream.git > >>>>>>> branch: origin/HEVC_UAPI_V4 > >>>>>>> > >>>>>>> Gstreamer: > >>>>>>> https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer.git > >>>>>>> branch: origin/benjamin.gaignard1/gstreamer-HEVC_aligned_with_kernel_5.15 > >>>>>>> > >>>>>>> > >>>>>>> I am still not able to run h.265/HEVC tests. > >>>>>> Hello Adam, > >>>>>> > >>>>>> I have updated the following branches with the versions I have used today: > >>>>>> > >>>>>> Kernel: > >>>>>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream.git > >>>>>> branch: origin/HEVC_UAPI_V5 only one change in documentation vs version 4 but rebased in v5.18-rc1 > >>>>>> > >>>>>> Gstreamer: > >>>>>> https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer.git > >>>>>> branch: origin/benjamin.gaignard1/gstreamer-HEVC_aligned_with_kernel_5.15 updated on the latest GST main branch > >>>>>> > >>>>>> I hope this will work fine this time. > >>>>> I wish I had better news for you: > >>>>> > >>>>> dmesg shows the hantro driver is being loaded: > >>>>> > >>>>> [ 38.612243] hantro-vpu 38300000.video-codec: registered > >>>>> nxp,imx8mq-vpu-g1-dec as /dev/video0 > >>>>> [ 38.612618] hantro-vpu 38310000.video-codec: registered > >>>>> nxp,imx8mq-vpu-g2-dec as /dev/video1 > >>>>> > >>>>> # gst-inspect-1.0 |grep 265 > >>>>> libav: avdec_h265: libav HEVC (High Efficiency Video Coding) decoder > >>>>> rtp: rtph265depay: RTP H265 depayloader > >>>>> rtp: rtph265pay: RTP H265 payloader > >>>>> typefindfunctions: video/x-h265: h265, x265, 265 > >>>>> v4l2codecs: v4l2slh265dec: V4L2 Stateless H.265 Video Decoder > >>>>> videoparsersbad: h265parse: H.265 parser > >>>>> > >>>>> Fluster reports: > >>>>> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for > >>>>> GStreamer 1.0... ❌ > >>>> Still the same error about non supported control ? > >>> # gst-lagst-launch-1.0 appsrc num-buffers=0 ! h265parse ! > >>> v4l2slh265dec ! fakesink > >>> Setting pipeline to PAUSED ... > >>> 0:00:01.704385508 420 0xaaaaec0a76c0 ERROR v4l2codecs-decoder > >>> gstv4l2decoder.c:725:gst_v4l2_decoder_get_controls:<v4l2decoder2> > >>> VIDIOC_G_EXT_CTRLS failed: Invalid argument > >>> ERROR: from element > >>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Driver did not > >>> report framing and start code method. > >>> Additional debug info: > >>> ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codech265dec.c(155): > >>> gst_v4l2_codec_h265_dec_open (): > >>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: > >>> gst_v4l2_decoder_get_controls() failed: Invalid argument > >>> ERROR: pipeline doesn't want to preroll. > >>> ERROR: from element > >>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Could not > >>> initialize supporting library. > >>> Additional debug info: > >>> ../subprojects/gst-plugins-base/gst-libs/gst/video/gstvideodecoder.c(2909): > >>> gst_video_decoder_change_state (): > >>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: > >>> Failed to open decoder > >>> ERROR: pipeline doesn't want to preroll. > >>> Failed to set pipeline to PAUSED. > >>> Setting pipeline to NULL ... > >>> Freeing pipeline ... > >> What is the result of v4l2-ctl ? > >> Here I have: > >> v4l2-ctl --all -d 1 > >> Driver Info: > >> Driver name : hantro-vpu > >> Card type : nxp,imx8mq-vpu-g2-dec > >> Bus info : platform: hantro-vpu > >> Driver version : 5.18.0 > >> Capabilities : 0x84204000 > >> Video Memory-to-Memory Multiplanar > >> Streaming > >> Extended Pix Format > >> Device Capabilities > >> Device Caps : 0x04204000 > >> Video Memory-to-Memory Multiplanar > >> Streaming > >> Extended Pix Format > >> Media Driver Info: > >> Driver name : hantro-vpu > >> Model : hantro-vpu > >> Serial : > >> Bus info : platform: hantro-vpu > >> Media version : 5.18.0 > >> Hardware revision: 0x00000000 (0) > >> Driver version : 5.18.0 > >> Interface Info: > >> ID : 0x0300000c > >> Type : V4L Video > >> Entity Info: > >> ID : 0x00000001 (1) > >> Name : nxp,imx8mq-vpu-g2-dec-source > >> Function : V4L2 I/O > >> Pad 0x01000002 : 0: Source > >> Link 0x02000008: to remote pad 0x1000004 of entity 'nxp,imx8mq-vpu-g2-dec-proc': Data, Enabled, Immutable > >> Priority: 2 > >> Format Video Capture Multiplanar: > >> Width/Height : 48/48 > >> Pixel Format : 'VT12' (Y/CbCr 4:2:0 (4x4 Linear)) > >> Field : None > >> Number of planes : 1 > >> Flags : > >> Colorspace : JPEG > >> Transfer Function : Default > >> YCbCr/HSV Encoding: Default > >> Quantization : Default > >> Plane 0 : > >> Bytes per Line : 48 > >> Size Image : 3600 > >> Format Video Output Multiplanar: > >> Width/Height : 48/48 > >> Pixel Format : 'S265' (HEVC Parsed Slice Data) > >> Field : None > >> Number of planes : 1 > >> Flags : > >> Colorspace : JPEG > >> Transfer Function : Default > >> YCbCr/HSV Encoding: Default > >> Quantization : Default > >> Plane 0 : > >> Bytes per Line : 0 > >> Size Image : 4608 > >> > >> User Controls > >> > >> hantro_hevc_slice_header_skip_b 0x00981a80 (int) : min=0 max=256 step=1 default=0 value=0 > >> > >> Codec Controls > >> > >> hevc_profile 0x00990b67 (menu) : min=0 max=2 default=0 value=0 > >> 0: Main > >> 1: Main Still Picture > >> 2: Main 10 > >> hevc_level 0x00990b68 (menu) : min=0 max=8 default=0 value=0 > >> 0: 1 > >> 1: 2 > >> 2: 2.1 > >> 3: 3 > >> 4: 3.1 > >> 5: 4 > >> 6: 4.1 > >> 7: 5 > >> 8: 5.1 > >> > >> Stateless Codec Controls > >> > >> vp9_frame_decode_parameters 0x00a40a2c (unknown): type=261 flags=has-payload > >> vp9_probabilities_updates 0x00a40a2d (unknown): type=260 flags=has-payload > >> hevc_sequence_parameter_set 0x00a40a90 (unknown): type=270 flags=has-payload > >> hevc_picture_parameter_set 0x00a40a91 (unknown): type=271 flags=has-payload > >> hevc_scaling_matrix 0x00a40a93 (unknown): type=273 flags=has-payload > >> hevc_decode_parameters 0x00a40a94 (unknown): type=274 flags=has-payload > >> hevc_decode_mode 0x00a40a95 (menu) : min=1 max=1 default=1 value=1 > >> 1: Frame-Based > >> hevc_start_code 0x00a40a96 (menu) : min=1 max=1 default=1 value=1 > >> 1: Annex B Start Code > >> > > Mine looks like: > > > > # v4l2-ctl --all -d 1 > > Driver Info: > > Driver name : hantro-vpu > > Card type : nxp,imx8mq-vpu-g2-dec > > Bus info : platform: hantro-vpu > > Driver version : 5.18.0 > > Capabilities : 0x84204000 > > Video Memory-to-Memory Multiplanar > > Streaming > > Extended Pix Format > > Device Capabilities > > Device Caps : 0x04204000 > > Video Memory-to-Memory Multiplanar > > Streaming > > Extended Pix Format > > Media Driver Info: > > Driver name : hantro-vpu > > Model : hantro-vpu > > Serial : > > Bus info : platform: hantro-vpu > > Media version : 5.18.0 > > Hardware revision: 0x00000000 (0) > > Driver version : 5.18.0 > > Interface Info: > > ID : 0x0300000c > > Type : V4L Video > > Entity Info: > > ID : 0x00000001 (1) > > Name : nxp,imx8mq-vpu-g2-dec-source > > Function : V4L2 I/O > > Pad 0x01000002 : 0: Source > > Link 0x02000008: to remote pad 0x1000004 of entity > > 'nxp,imx8mq-vpu-g2-dec-proc': Data, Enabled, Immutable > > Priority: 2 > > Format Video Capture Multiplanar: > > Width/Height : 48/48 > > Pixel Format : 'VT12' (Y/CbCr 4:2:0 (4x4 Linear)) > > Field : None > > Number of planes : 1 > > Flags : > > Colorspace : JPEG > > Transfer Function : Default > > YCbCr/HSV Encoding: Default > > Quantization : Default > > Plane 0 : > > Bytes per Line : 48 > > Size Image : 3600 > > Format Video Output Multiplanar: > > Width/Height : 48/48 > > Pixel Format : 'S265' (HEVC Parsed Slice Data) > > Field : None > > Number of planes : 1 > > Flags : > > Colorspace : JPEG > > Transfer Function : Default > > YCbCr/HSV Encoding: Default > > Quantization : Default > > Plane 0 : > > Bytes per Line : 0 > > Size Image : 4608 > > > > User Controls > > > > hantro_hevc_slice_header_skip_b 0x00981a80 (int) : min=0 max=256 > > step=1 default=0 value=0 > > > > Codec Controls > > > > hevc_profile 0x00990b67 (menu) : min=0 max=2 > > default=0 value=0 > > 0: Main > > 1: Main Still Picture > > 2: Main 10 > > hevc_level 0x00990b68 (menu) : min=0 max=8 > > default=0 value=0 > > 0: 1 > > 1: 2 > > 2: 2.1 > > 3: 3 > > 4: 3.1 > > 5: 4 > > 6: 4.1 > > 7: 5 > > 8: 5.1 > > > > Stateless Codec Controls > > > > vp9_frame_decode_parameters 0x00a40a2c (unknown): type=261 flags=has-payload > > vp9_probabilities_updates 0x00a40a2d (unknown): type=260 flags=has-payload > > hevc_sequence_parameter_set 0x00a40a90 (unknown): type=270 flags=has-payload > > hevc_picture_parameter_set 0x00a40a91 (unknown): type=271 flags=has-payload > > hevc_scaling_matrix 0x00a40a93 (unknown): type=273 flags=has-payload > > hevc_decode_parameters 0x00a40a94 (unknown): type=274 flags=has-payload > > hevc_decode_mode 0x00a40a95 (menu) : min=1 max=1 > > default=1 value=1 > > 1: Frame-Based > > hevc_start_code 0x00a40a96 (menu) : min=1 max=1 > > default=1 value=1 > > 1: Annex B Start Code > > It is the same so that suggest the issue is coming from GStreamer plugin. Do I need to build Gtreamer in a special way? I just invoked meson then ninja, but I didn't pass any additional parameters other than the build directory name. adam > > > > >>>> Benjamin > >>>> > >>>>> adam > >>>>>> Benjamin > >>>>>> > >>>>>>> adam > >>>>>>>> adam > >>>>>>>>> Regards, > >>>>>>>>> Benjamin > >>>>>>>>> > >>>>>>>>>>> adam > >>>>>>>>>>>> Thanks, > >>>>>>>>>>>> Benjamin > >>>>>>>>>>>> > >>>>>>>>>>>>> Benjamin > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> Benjamin Gaignard (12): > >>>>>>>>>>>>> media: uapi: HEVC: Add missing fields in HEVC controls > >>>>>>>>>>>>> media: uapi: HEVC: Rename HEVC stateless controls with STATELESS > >>>>>>>>>>>>> prefix > >>>>>>>>>>>>> media: uapi: HEVC: Add document uAPI structure > >>>>>>>>>>>>> media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS > >>>>>>>>>>>>> as a > >>>>>>>>>>>>> dynamic array > >>>>>>>>>>>>> media: uapi: Move parsed HEVC pixel format out of staging > >>>>>>>>>>>>> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS > >>>>>>>>>>>>> control > >>>>>>>>>>>>> media: uapi: Move the HEVC stateless control type out of staging > >>>>>>>>>>>>> media: controls: Log HEVC stateless control in .std_log > >>>>>>>>>>>>> media: uapi: Create a dedicated header for Hantro control > >>>>>>>>>>>>> media: uapi: HEVC: fix padding in v4l2 control structures > >>>>>>>>>>>>> media: uapi: Change data_bit_offset definition > >>>>>>>>>>>>> media: uapi: move HEVC stateless controls out of staging > >>>>>>>>>>>>> > >>>>>>>>>>>>> Hans Verkuil (3): > >>>>>>>>>>>>> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY > >>>>>>>>>>>>> v4l2-ctrls: add support for dynamically allocated arrays. > >>>>>>>>>>>>> vivid: add dynamic array test control > >>>>>>>>>>>>> > >>>>>>>>>>>>> .../userspace-api/media/drivers/hantro.rst | 5 - > >>>>>>>>>>>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 833 > >>>>>>>>>>>>> ++++++++++++++++++ > >>>>>>>>>>>>> .../media/v4l/ext-ctrls-codec.rst | 780 > >>>>>>>>>>>>> ---------------- > >>>>>>>>>>>>> .../media/v4l/pixfmt-compressed.rst | 7 +- > >>>>>>>>>>>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 20 + > >>>>>>>>>>>>> .../media/v4l/vidioc-queryctrl.rst | 8 + > >>>>>>>>>>>>> .../media/videodev2.h.rst.exceptions | 5 + > >>>>>>>>>>>>> .../media/test-drivers/vivid/vivid-ctrls.c | 15 + > >>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++- > >>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++- > >>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 37 +- > >>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +- > >>>>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +- > >>>>>>>>>>>>> drivers/staging/media/hantro/hantro_drv.c | 27 +- > >>>>>>>>>>>>> drivers/staging/media/hantro/hantro_hevc.c | 8 +- > >>>>>>>>>>>>> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +- > >>>>>>>>>>>>> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +- > >>>>>>>>>>>>> .../staging/media/sunxi/cedrus/cedrus_h265.c | 2 +- > >>>>>>>>>>>>> include/media/hevc-ctrls.h | 250 ------ > >>>>>>>>>>>>> include/media/v4l2-ctrls.h | 48 +- > >>>>>>>>>>>>> include/uapi/linux/hantro-media.h | 19 + > >>>>>>>>>>>>> include/uapi/linux/v4l2-controls.h | 439 +++++++++ > >>>>>>>>>>>>> include/uapi/linux/videodev2.h | 13 + > >>>>>>>>>>>>> 23 files changed, 1697 insertions(+), 1170 deletions(-) > >>>>>>>>>>>>> delete mode 100644 include/media/hevc-ctrls.h > >>>>>>>>>>>>> create mode 100644 include/uapi/linux/hantro-media.h > >>>>>>>>>>>>>
