Hi All.
I'm currently doing battle with the stateful video decoder API for
video decode on the Raspberry Pi.
Reading the descriptive docs[1] there is no obligation to
STREAMON(CAPTURE) before feeding in OUTPUT buffers and waiting for
V4L2_EVENT_SOURCE_CHANGE to configure the CAPTURE queue. Great! It
makes my colleague who is working on the userspace side happy as it
saves a config step of allocating buffers that are never needed.
I have been using the v4l2_mem2mem framework, same as some other
decoders. We use v4l2_m2m in the buffered mode as it's actually
remoted over to the VPU via the MMAL API, and so the src and dest are
asynchronous from V4L2's perspective.
Said colleague then complained that he couldn't follow the flow
described in the docs linked above as it never produced the
V4L2_EVENT_SOURCE_CHANGE event.
Digging into it, it's the v4l2_mem2mem framework stopping me.
__v4l2_m2m_try_queue[2] has
if (!m2m_ctx->out_q_ctx.q.streaming
|| !m2m_ctx->cap_q_ctx.q.streaming) {
dprintk("Streaming needs to be on for both queues\n");
return;
}
So I'm never going to get any of the OUTPUT buffers even queued to my
driver until STREAMON(CAPTURE). That contradicts the documentation :-(
Now I can see that on a non-buffered M2M device you have to have both
OUTPUT and CAPTURE enabled because it wants to produce a CAPTURE
buffer for every OUTPUT buffer on a 1:1 basis. On a buffered codec
tweaking that one clause to
if (!m2m_ctx->out_q_ctx.buffered &&
(!m2m_ctx->out_q_ctx.q.streaming ||
!m2m_ctx->cap_q_ctx.q.streaming)) {
solves the problem, but is that a generic solution? I don't have any
other platforms to test against.
However it poses a larger question for my colleague as to what
behaviour he can rely on in userspace. Is there a way for userspace to
know whether it is permitted on a specific codec implementation to
follow the docs and not STREAMON(CAPTURE) until
V4L2_EVENT_SOURCE_CHANGE? If not then the documentation is invalid for
many devices.
On a very brief survey of a few existing drivers I see:
- the Coda driver uses v4l2_m2m in the same mode as my driver, so
presumably it's not currently going to be following the documentation
but the above change should make it do so.
- meson/vdec doesn't work in buffered mode, so it's never going to be
able to follow the docs. There is a TODO stating that (although
implying it's mainly on MPEG1 & 2).
- mtk-vcodec is also not in buffered mode, and appears to have
secondary checks that it has both a src and dst buffer before passing
anything to the hardware. Is there any way for that to be able to
follow the docs?
Guidance please.
Thanks
Dave
[1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/dev-decoder.html#initialization
[2] https://git.linuxtv.org/media_tree.git/tree/drivers/media/v4l2-core/v4l2-mem2mem.c#n303
