Hi Ming and Nicolas On Mon, 4 Jul 2022 at 16:53, Nicolas Dufresne <nicolas@xxxxxxxxxxxx> wrote: > > Le mardi 28 juin 2022 à 10:19 +0800, Ming Qian a écrit : > > By setting the V4L2_BUF_FLAG_CODECCONFIG flag, > > user-space should be able to hint decoder > > the vb2 only contains codec config header, > > but does not contain any frame data. > > It's only used for parsing header, and can't be decoded. > > This is copied from OMX specification. I think we we import this, we should at > least refer to the original. > > > > > Current, it's usually used by android. > > > > Signed-off-by: Ming Qian <ming.qian@xxxxxxx> > > --- > > Documentation/userspace-api/media/v4l/buffer.rst | 9 +++++++++ > > include/uapi/linux/videodev2.h | 2 ++ > > 2 files changed, 11 insertions(+) > > > > diff --git a/Documentation/userspace-api/media/v4l/buffer.rst b/Documentation/userspace-api/media/v4l/buffer.rst > > index 4638ec64db00..acdc4556f4f4 100644 > > --- a/Documentation/userspace-api/media/v4l/buffer.rst > > +++ b/Documentation/userspace-api/media/v4l/buffer.rst > > @@ -607,6 +607,15 @@ Buffer Flags > > the format. Any subsequent call to the > > :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>` ioctl will not block anymore, > > but return an ``EPIPE`` error code. > > + * .. _`V4L2-BUF-FLAG-CODECCONFIG`: > > + > > + - ``V4L2_BUF_FLAG_CODECCONFIG`` > > + - 0x00200000 > > + - This flag may be set when the buffer only contains codec config > > + header, but does not contain any frame data. Usually the codec config > > + header is merged to the next idr frame, with the flag > > + ``V4L2_BUF_FLAG_KEYFRAME``, but there is still some scenes that will > > + split the header and queue it separately. > > I think the documentation is clear. Now, if a driver uses this, will existing > userland (perhaps good to check GStreamer, FFMPEG and Chromium ?) will break ? > So we need existing driver to do this when flagged to, and just copy/append when > the userland didn't opt-in that feature ? The commit text says it is for userspace feeding data into a video decoder, so it's a userspace choice instead of driver. For encoders there is already V4L2_CID_MPEG_VIDEO_HEADER_MODE [1] which allows for V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE or V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME. FFmpeg selects _SEPARATE by default [2], whilst the default is normally _JOINED_WITH_1ST_FRAME. It does raise the question as to whether all decoders will support header byte only buffers, and does there need to be a capabilities flag to denote that it is supported. And should encoders in V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE mode set it on the headers only buffers? A number of undefined elements of how this should be implemented/used :-( Dave [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/ext-ctrls-codec.html [2] https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/v4l2_m2m_enc.c#L196 > > * .. _`V4L2-BUF-FLAG-REQUEST-FD`: > > > > - ``V4L2_BUF_FLAG_REQUEST_FD`` > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > > index 5311ac4fde35..8708ef257710 100644 > > --- a/include/uapi/linux/videodev2.h > > +++ b/include/uapi/linux/videodev2.h > > @@ -1131,6 +1131,8 @@ static inline __u64 v4l2_timeval_to_ns(const struct timeval *tv) > > #define V4L2_BUF_FLAG_TSTAMP_SRC_SOE 0x00010000 > > /* mem2mem encoder/decoder */ > > #define V4L2_BUF_FLAG_LAST 0x00100000 > > +/* Buffer only contains codec header */ > > +#define V4L2_BUF_FLAG_CODECCONFIG 0x00200000 > > /* request_fd is valid */ > > #define V4L2_BUF_FLAG_REQUEST_FD 0x00800000 > > >
