Le mercredi 22 avril 2020 à 14:43 +0300, Stanimir Varbanov a écrit : > Hi Nicolas, > > On 4/21/20 1:09 AM, Nicolas Dufresne wrote: > > Hi Stanimir, > > > > Le lundi 20 avril 2020 à 17:38 +0300, Stanimir Varbanov a écrit : > > > Hi, > > > > > > I need to port a decoder v4l2 control in mainline Venus driver which > > > instructs the decoder to decode sync frames only (I frame/ IDR frame). > > > In practice the usage of such control is to generate thumbnails for > > > particular video. > > > > > > To do that I researched what we have currently in v4l2-controls.h and > > > found something similar but for encoders: > > > > > > V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME > > > "Force a key frame for the next queued buffer. Applicable to encoders. > > > This is a general, codec-agnostic keyframe control." > > > > > > I think I have two options: > > > > > > 1. reuse V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME for the decoder and document it > > > > > > 2. create a new v4l control V4L2_CID_MPEG_VIDC_VIDEO_SYNC_FRAME_DECODE > > > > Seems analogue to GStreamer flag GST_SEEK_FLAG_TRICKMODE_KEY_UNIT, if that helps > > your inspiration. It's used to speed up fast-forward (ore thumbnails > > generation). What the decoder will effectively do is to skip the P and the B > > frames. Some decoder offers fine grain control on that. > > > > That being said, in GStreamer (and probably most framework) frames are skipped > > in userspace. But decoder tend to wait for the next key frame to output the > > first one, so we endup having to do more work to force the frame out (like > > CMD_STOP/START cycle, or crafts an early AUD). Would be nice to document if that > > mode will help in this regard (or not). > > > > For protocols like DASH, when doing fast-forward we don't even download the non- > > key-frames from the server, so the skipping behaviour does not provide any gain > > any way. > > > > > Any suggestions? > > > > > > > Could be as Hans suggested, or something less generic but that covers more > > trickmodes ? > > > > V4L2_CID_MPEG_VIDEO_DECODER_H264_TRICKMODE > > - No skip > > - Skip B Frames > > - I frame only > > > > Or maybe try and construct something base on what other do ? > > > > FFMPEG software decoder has the following: > > (0): Skip nothing - 0 > > (1): Skip B-frames - 1 > > (2): Skip IDCT/Dequantization - 2 > > (5): Skip everything - 5 > > > > CODA from Chips&Media is fancier, and propose some variants that have > > implication how muchs state is maintained, hence allowing or not the transition > > back to non-skipping outside IRAP boundary. > > NORMAL > > THUMBNAIL > > Skip nonIRAP > > Skip nonREF-PIC > > Skip Temporal Layer > > > > I didn't find such feature for Samsung MFC, but Amlogic is likely one candidate > > having this type of skipping feature (as it also has HW demuxers). Hope this > > helps getting the big picture, hence proposing uAPI that isn't too limited. > > > > regards, > > Nicolas > > > > I like your idea to have more generic one and codec-agnostic. I guess > something like below menu control: > > V4L2_CID_MPEG_VIDEO_DECODER_DECODE_PICTURE_TYPE > "Normal" - decode everything applicable to the codec (default) > "Thumbnail" - decode sync frames (pictures?) with low memory usage > "Sync frames" - decode sync frames without memory constraints > "Skip B frames" - decode sync frames and unidirectional frames > "Skip Temporal layer(s)" - decode everything but skip temporal layers > above base layer That's promising, of course it will be hard to claim that "Skip B- frame" is codec-agnostic. But it could be acceptable if we come up with a proper story for enumeration. If I may make the analogy with the enum-framerate, the answer from that can vary depending on the configured format. The enumeration of the menu could behave similarly I think. Nicolas
