Re: decode sync frames only

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Le mercredi 22 avril 2020 à 14:43 +0300, Stanimir Varbanov a écrit :
> Hi Nicolas,
> 
> On 4/21/20 1:09 AM, Nicolas Dufresne wrote:
> > Hi Stanimir,
> > 
> > Le lundi 20 avril 2020 à 17:38 +0300, Stanimir Varbanov a écrit :
> > > Hi,
> > > 
> > > I need to port a decoder v4l2 control in mainline Venus driver which
> > > instructs the decoder to decode sync frames only (I frame/ IDR frame).
> > > In practice the usage of such control is to generate thumbnails for
> > > particular video.
> > > 
> > > To do that I researched what we have currently in v4l2-controls.h and
> > > found something similar but for encoders:
> > > 
> > > V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME
> > > "Force a key frame for the next queued buffer. Applicable to encoders.
> > > This is a general, codec-agnostic keyframe control."
> > > 
> > > I think I have two options:
> > > 
> > > 1. reuse V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME for the decoder and document it
> > > 
> > > 2. create a new v4l control V4L2_CID_MPEG_VIDC_VIDEO_SYNC_FRAME_DECODE
> > 
> > Seems analogue to GStreamer flag GST_SEEK_FLAG_TRICKMODE_KEY_UNIT, if that helps
> > your inspiration. It's used to speed up fast-forward (ore thumbnails
> > generation). What the decoder will effectively do is to skip the P and the B
> > frames. Some decoder offers fine grain control on that.
> > 
> > That being said, in GStreamer (and probably most framework) frames are skipped
> > in userspace. But decoder tend to wait for the next key frame to output the
> > first one, so we endup having to do more work to force the frame out (like
> > CMD_STOP/START cycle, or crafts an early AUD). Would be nice to document if that
> > mode will help in this regard (or not). 
> > 
> > For protocols like DASH, when doing fast-forward we don't even download the non-
> > key-frames from the server, so the skipping behaviour does not provide any gain
> > any way.
> > 
> > > Any suggestions?
> > > 
> > 
> > Could be as Hans suggested, or something less generic but that covers more
> > trickmodes ?
> > 
> > V4L2_CID_MPEG_VIDEO_DECODER_H264_TRICKMODE
> >   - No skip
> >   - Skip B Frames
> >   - I frame only
> > 
> > Or maybe try and construct something base on what other do ?
> > 
> > FFMPEG software decoder has the following:
> >   (0): Skip nothing     - 0
> >   (1): Skip B-frames    - 1
> >   (2): Skip IDCT/Dequantization - 2
> >   (5): Skip everything  - 5
> > 
> > CODA from Chips&Media is fancier, and propose some variants that have
> > implication how muchs state is maintained, hence allowing or not the transition
> > back to non-skipping outside IRAP boundary.
> >   NORMAL
> >   THUMBNAIL
> >   Skip nonIRAP
> >   Skip nonREF-PIC
> >   Skip Temporal Layer
> > 
> > I didn't find such feature for Samsung MFC, but Amlogic is likely one candidate
> > having this type of skipping feature (as it also has HW demuxers). Hope this
> > helps getting the big picture, hence proposing uAPI that isn't too limited.
> > 
> > regards,
> > Nicolas
> > 
> 
> I like your idea to have more generic one and codec-agnostic. I guess
> something like below menu control:
> 
> V4L2_CID_MPEG_VIDEO_DECODER_DECODE_PICTURE_TYPE
> "Normal"	- decode everything applicable to the codec (default)
> "Thumbnail"	- decode sync frames (pictures?) with low memory usage
> "Sync frames"	- decode sync frames without memory constraints
> "Skip B frames"	- decode sync frames and unidirectional frames
> "Skip Temporal layer(s)" - decode everything but skip temporal layers
> above base layer

That's promising, of course it will be hard to claim that "Skip B-
frame" is codec-agnostic. But it could be acceptable if we come up with
a proper story for enumeration. If I may make the analogy with the
enum-framerate, the answer from that can vary depending on the
configured format. The enumeration of the menu could behave similarly I
think.

Nicolas




[Index of Archives]     [Linux Input]     [Video for Linux]     [Gstreamer Embedded]     [Mplayer Users]     [Linux USB Devel]     [Linux Audio Users]     [Linux Kernel]     [Linux SCSI]     [Yosemite Backpacking]

  Powered by Linux