Hi Nicolas, On 4/21/20 1:09 AM, Nicolas Dufresne wrote: > Hi Stanimir, > > Le lundi 20 avril 2020 à 17:38 +0300, Stanimir Varbanov a écrit : >> Hi, >> >> I need to port a decoder v4l2 control in mainline Venus driver which >> instructs the decoder to decode sync frames only (I frame/ IDR frame). >> In practice the usage of such control is to generate thumbnails for >> particular video. >> >> To do that I researched what we have currently in v4l2-controls.h and >> found something similar but for encoders: >> >> V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME >> "Force a key frame for the next queued buffer. Applicable to encoders. >> This is a general, codec-agnostic keyframe control." >> >> I think I have two options: >> >> 1. reuse V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME for the decoder and document it >> >> 2. create a new v4l control V4L2_CID_MPEG_VIDC_VIDEO_SYNC_FRAME_DECODE > > Seems analogue to GStreamer flag GST_SEEK_FLAG_TRICKMODE_KEY_UNIT, if that helps > your inspiration. It's used to speed up fast-forward (ore thumbnails > generation). What the decoder will effectively do is to skip the P and the B > frames. Some decoder offers fine grain control on that. > > That being said, in GStreamer (and probably most framework) frames are skipped > in userspace. But decoder tend to wait for the next key frame to output the > first one, so we endup having to do more work to force the frame out (like > CMD_STOP/START cycle, or crafts an early AUD). Would be nice to document if that > mode will help in this regard (or not). > > For protocols like DASH, when doing fast-forward we don't even download the non- > key-frames from the server, so the skipping behaviour does not provide any gain > any way. > >> >> Any suggestions? >> > > Could be as Hans suggested, or something less generic but that covers more > trickmodes ? > > V4L2_CID_MPEG_VIDEO_DECODER_H264_TRICKMODE > - No skip > - Skip B Frames > - I frame only > > Or maybe try and construct something base on what other do ? > > FFMPEG software decoder has the following: > (0): Skip nothing - 0 > (1): Skip B-frames - 1 > (2): Skip IDCT/Dequantization - 2 > (5): Skip everything - 5 > > CODA from Chips&Media is fancier, and propose some variants that have > implication how muchs state is maintained, hence allowing or not the transition > back to non-skipping outside IRAP boundary. > NORMAL > THUMBNAIL > Skip nonIRAP > Skip nonREF-PIC > Skip Temporal Layer > > I didn't find such feature for Samsung MFC, but Amlogic is likely one candidate > having this type of skipping feature (as it also has HW demuxers). Hope this > helps getting the big picture, hence proposing uAPI that isn't too limited. > > regards, > Nicolas > I like your idea to have more generic one and codec-agnostic. I guess something like below menu control: V4L2_CID_MPEG_VIDEO_DECODER_DECODE_PICTURE_TYPE "Normal" - decode everything applicable to the codec (default) "Thumbnail" - decode sync frames (pictures?) with low memory usage "Sync frames" - decode sync frames without memory constraints "Skip B frames" - decode sync frames and unidirectional frames "Skip Temporal layer(s)" - decode everything but skip temporal layers above base layer -- regards, Stan
