[RFC] V4L2 unified low-level decoder API

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

This RFC aims to start discussions in order to define the codec specific 
controls structures to fulfill the low-level decoder API needed by non 
"Stream API" based decoders ("stateless" or "Frame API" based decoders).
Several implementation exists now which runs on several SoC and various 
software frameworks.
The idea is to find the communalities between all those implementations 
and SoC to define a single unified interface in V4L2 includes.
Even if "Request API" is needed to pass those codec specific controls 
from userspace down to kernel on a per-buffer basis, we can start 
discussions and define the controls in parallel of its development.
We can even propose some implementations based on existing V4L2 control 
framework (which doesn't support "per-frame" basis) by ensuring 
atomicity of sequence S_EXT_CTRL(header[i])/QBUF(stream[i]). Constraint 
can then be relaxed when "Request API" is merged.

I would like to propose to work on a "per-codec" basis, having at least 
2 different SoC and 2 different frameworks to test and validate controls.
To do so, I have tried to identify some people that have worked on this 
subject and have proposed some implementations, feel free to correct me 
and enhance the list if needed:
* MPEG2/MPEG4
    - Florent Revest for Allwinner A13 CedarX support [1] tested with 
VLC -> libVA + sunxi-cedrus-drv-video -> V4L2
    - Myself for STMicroelectronics Delta support [2] tested with 
GStreamer V4L2 -> libv4l2 + libv4l-delta plugin -> V4L2

* VP8
- Pawel Osciak for Rockchip RK3288, RK3399? VPU Support [3] tested with 
Chromium -> V4L2
- Jung Zhao for Rockchip RK3288 VPU support [4] <cannot find the 
framework used>

* H264
- Pawel Osciak for Rockchip RK3288, RK3399? VPU Support [5] tested with 
Chromium -> V4L2
- Randy Li for Rockchip RK3288  VPU support [6] tested with VLC? -> 
libVA + rockchip-va-driver -> V4L2
VLC? -> libVDPAU + rockchip-va-driver -> V4L2

I can work to define MPEG2/MPEG4 controls and propose functional 
implementations for those codecs, and will be glad to co-work with you 
Florent.
I can help on H264 on a code review basis based on the functional H264 
setup I have in-house and codec knowledge, but I cannot provide 
implementation in a reasonable timeframe, same for VP8.

Apart of very details of each codec, we have also to state about generic 
concerns such as:
- new pixel format introduction (VP8 => VP8F, H264 => S264, MPG2 => 
MG2F, MPG4 => MG4F)
- new device caps to indicate that driver requires extra headers ? maybe 
not needed because redundant with new pixel format
- continue to modify v4l2-controls.h ? or do we add some new specific 
header files (H264 is huge!) ?
- how to manage sequence header & picture header, optional/extended 
controls (MPEG2 sequence/picture extensions, H264 SEI, ...). Personally 
I have added flags inside a single control structure, H264 is done in a 
different way using several controls (SPS/PPS/SLICE/DECODE/...)

Thanks you to all of you for your attention and feel free to react on 
this topic if you are interested to work on this subject.

Best regards,
Hugues.

[0] [ANN] Codec & Request API Brainstorm meeting Oct 10 & 
11https://www.spinics.net/lists/linux-media/msg106699.html
[1] MPEG2 A13 CedarXhttp://www.spinics.net/lists/linux-media/msg104823.html
[1] MPEG4 A13 CedarXhttp://www.spinics.net/lists/linux-media/msg104817.html
[2] MPEG2 STi4xx 
Deltahttp://www.spinics.net/lists/linux-media/msg106240.html
[2] MPEG4 STi4xx Delta is also supported but not yet pushed
[3] VP8 Rockchip RK3288, RK3399? 
VPUhttps://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/refs/heads/master/sys-kernel/linux-headers/files/0002-CHROMIUM-v4l-Add-VP8-low-level-decoder-API-controls.patch 
[4] VP8 Rockchip RK3288 
VPUhttp://www.spinics.net/lists/linux-media/msg97997.html
[5] H264 Rockchip RK3288, RK3399? 
VPUhttps://chromium.googlesource.com/chromiumos/overlays/chromiumos-overlay/+/refs/heads/master/sys-kernel/linux-headers/files/0001-CHROMIUM-media-headers-Import-V4L2-headers-from-Chro.patch
[6] H264 Rockchip RK3288 
VPUhttp://www.spinics.net/lists/linux-media/msg105095.html
��.n��������+%������w��{.n�����{��g����^n�r������&��z�ޗ�zf���h���~����������_��+v���)ߣ�

[Index of Archives]     [Linux Input]     [Video for Linux]     [Gstreamer Embedded]     [Mplayer Users]     [Linux USB Devel]     [Linux Audio Users]     [Linux Kernel]     [Linux SCSI]     [Yosemite Backpacking]
  Powered by Linux