Hi, Here is an update regarding MPEG-2 implementation based on ST video decoder: * MPEG-2 API + DELTA kernel driver: http://www.mail-archive.com/linux-media@xxxxxxxxxxxxxxx/msg107405.html * libv4l-codecparsers plugin including MPEG-2 back-end: http://www.mail-archive.com/linux-media@xxxxxxxxxxxxxxx/msg107812.html Please note that this is implemented & functional using currently available V4L2 control framework (no Request API), assuming that user side keeps unicity of S_EXT_CTRL() / QBUF(OUTPUT) pair. Request API will remove this constraint, but the point here is to define control interface, as far as I have understood code, Request API will not affect those control definitions. Some updates inline thereafter regarding activities on this subject; me for MPEG-2 on ST platform and Randy Li, Nicolas Dufresne, Ayaka for H264 on Rockchip platform: On 11/14/2016 10:55 AM, Hans Verkuil wrote: > On 10/27/2016 09:42 AM, Hugues FRUCHET wrote: >> Hi, >> >> This RFC aims to start discussions in order to define the codec specific >> controls structures to fulfill the low-level decoder API needed by non >> "Stream API" based decoders ("stateless" or "Frame API" based decoders). > > Let's refer to this as 'stateful' decoders and 'stateless' decoders. This > is the preferred terminology (and much more descriptive than 'Stream' vs > 'Frame'). It's also not really a new API, although it does rely on the > Request API. > >> Several implementation exists now which runs on several SoC and various >> software frameworks. >> The idea is to find the communalities between all those implementations >> and SoC to define a single unified interface in V4L2 includes. >> Even if "Request API" is needed to pass those codec specific controls >> from userspace down to kernel on a per-buffer basis, we can start >> discussions and define the controls in parallel of its development. >> We can even propose some implementations based on existing V4L2 control >> framework (which doesn't support "per-frame" basis) by ensuring >> atomicity of sequence S_EXT_CTRL(header[i])/QBUF(stream[i]). Constraint >> can then be relaxed when "Request API" is merged. >> >> I would like to propose to work on a "per-codec" basis, having at least >> 2 different SoC and 2 different frameworks to test and validate controls. >> To do so, I have tried to identify some people that have worked on this >> subject and have proposed some implementations, feel free to correct me >> and enhance the list if needed: >> * MPEG2/MPEG4 >> - Florent Revest for Allwinner A13 CedarX support [1] tested with >> VLC -> libVA + sunxi-cedrus-drv-video -> V4L2 >> - Myself for STMicroelectronics Delta support [2] tested with >> GStreamer V4L2 -> libv4l2 + libv4l-delta plugin -> V4L2 Available on ST platform with [2] & [2.1] patchset series. >> >> * VP8 >> - Pawel Osciak for Rockchip RK3288, RK3399? VPU Support [3] tested with >> Chromium -> V4L2 >> - Jung Zhao for Rockchip RK3288 VPU support [4] <cannot find the >> framework used> >> >> * H264 >> - Pawel Osciak for Rockchip RK3288, RK3399? VPU Support [5] tested with >> Chromium -> V4L2 >> - Randy Li for Rockchip RK3288 VPU support [6] tested with VLC? -> >> libVA + rockchip-va-driver -> V4L2 >> VLC? -> libVDPAU + rockchip-va-driver -> V4L2 Tested with Gstreamer -> VA-API element -> Rockchip VA-API driver -> V4L2 https://github.com/rockchip-linux/libvdpau-rockchip Study on-going for H264 userland/kernel partitioning in this thread: Request API: stateless VPU: the buffer mechanism and DPB management: http://www.mail-archive.com/linux-media@xxxxxxxxxxxxxxx/msg107165.html >> >> I can work to define MPEG2/MPEG4 controls and propose functional >> implementations for those codecs, and will be glad to co-work with you >> Florent. >> I can help on H264 on a code review basis based on the functional H264 >> setup I have in-house and codec knowledge, but I cannot provide >> implementation in a reasonable timeframe, same for VP8. >> >> Apart of very details of each codec, we have also to state about generic >> concerns such as: >> - new pixel format introduction (VP8 => VP8F, H264 => S264, MPG2 => >> MG2F, MPG4 => MG4F) >> - new device caps to indicate that driver requires extra headers ? maybe >> not needed because redundant with new pixel format > > That's indeed typically signaled through the pixelformat. > [2] is implemented this way in [media] v4l: add parsed MPEG-2 support, two new pixel format MG1P and MG2P ("P" for "Parsed") have been introduced: +#define V4L2_PIX_FMT_MPEG1_PARSED v4l2_fourcc('M', 'G', '1', 'P') /* MPEG1 with parsing metadata given through controls */ +#define V4L2_PIX_FMT_MPEG2_PARSED v4l2_fourcc('M', 'G', '2', 'P') /* MPEG2 with parsing metadata given through controls */ libv4l plugin [2.1] intercepts the pixel format information negotiated between user and driver (enum_fmt/try_fmt/get_fmt/s_fmt) and selects the right parser to use to convert MPEG-2 compressed format to new "MPEG-2 parsed" compressed format required by kernel driver. Plugin is designed to support several formats. >> - continue to modify v4l2-controls.h ? or do we add some new specific >> header files (H264 is huge!) ? >> - how to manage sequence header & picture header, optional/extended >> controls (MPEG2 sequence/picture extensions, H264 SEI, ...). Personally >> I have added flags inside a single control structure, H264 is done in a >> different way using several controls (SPS/PPS/SLICE/DECODE/...) >> >> Thanks you to all of you for your attention and feel free to react on >> this topic if you are interested to work on this subject. > > As long as the V4L2 driver underpins the various solutions I am happy :-) > > I do think that having a libv4l plugin will be useful since it will make > it easy for applications that support the stateful decoder to use the > same code for a stateless decoder by seamlessly using the plugin. > > This does not prevent other approaches at the same time, of course. > > Regards, > > Hans > [2] implements new extended controls for MPEG-2 in [media] v4l: add parsed MPEG-2 support http://www.mail-archive.com/linux-media@xxxxxxxxxxxxxxx/msg107406.html This has been done in uapi/linux/v4l2-controls.h as extension of already existing MPEG video controls, defining one control per "header" (so not using a single control with selection flag): #define V4L2_CID_MPEG_VIDEO_MPEG2_SEQ_HDR #define V4L2_CID_MPEG_VIDEO_MPEG2_SEQ_EXT #define V4L2_CID_MPEG_VIDEO_MPEG2_SEQ_DISPLAY_EXT #define V4L2_CID_MPEG_VIDEO_MPEG2_SEQ_MATRIX_EXT #define V4L2_CID_MPEG_VIDEO_MPEG2_PIC_HDR #define V4L2_CID_MPEG_VIDEO_MPEG2_PIC_EXT Those controls and their associated data structure have been defined based on MPEG-2 standard ISO/IEC 13818-2. >> >> Best regards, >> Hugues. >> >> [0] [ANN] Codec & Request API Brainstorm meeting Oct 10 & >> 11https://www.spinics.net/lists/linux-media/msg106699.html >> [1] MPEG2 A13 CedarXhttp://www.spinics.net/lists/linux-media/msg104823.html >> [1] MPEG4 A13 CedarXhttp://www.spinics.net/lists/linux-media/msg104817.html >> [2] MPEG2 STi4xx >> Deltahttp://www.spinics.net/lists/linux-media/msg106240.html [2] MPEG2 DELTA kernel driver: http://www.mail-archive.com/linux- media@xxxxxxxxxxxxxxx/msg107405.html [2.1] MPEG2 libv4l plugin: http://www.mail-archive.com/linux-media@xxxxxxxxxxxxxxx/msg107812.html >> [2] MPEG4 STi4xx Delta is also supported but not yet pushed >> [3] VP8 Rockchip RK3288, RK3399? >> VPUhttps://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/refs/heads/master/sys-kernel/linux-headers/files/0002-CHROMIUM-v4l-Add-VP8-low-level-decoder-API-controls.patch >> [4] VP8 Rockchip RK3288 >> VPUhttp://www.spinics.net/lists/linux-media/msg97997.html >> [5] H264 Rockchip RK3288, RK3399? >> VPUhttps://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/refs/heads/master/sys-kernel/linux-headers/files/0001-CHROMIUM-media-headers-Import-V4L2-headers-from-Chro.patch >> [6] H264 Rockchip RK3288 >> VPUhttp://www.spinics.net/lists/linux-media/msg105095.html https://github.com/rockchip-linux/libvdpau-rockchip https://github.com/rockchip-linux/gstreamer-rockchip https://github.com/rockchip-linux/mpp >> N�����r��y���b�X��ǧv�^�){.n�+����{���bj)���w* jg��� �����ݢj/���z�ޖ��2�ޙ���&�)ߡ�a���� �G���h��j:+v���w�٥ >>��.n��������+%������w��{.n�����{��g����^n�r������&��z�ޗ�zf���h���~����������_��+v���)ߣ�
