Hi Detlev,
On Wed, 19 Jun 2024 10:57:19 -0400, Detlev Casanova wrote:
>+ if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
>+ height *= 2;
>+
>+ if (width > ctx->coded_fmt.fmt.pix_mp.width ||
>+ height > ctx->coded_fmt.fmt.pix_mp.height)
>+ return -EINVAL;
I did further invesatigation on chromium. I find that before real video
is decoded, chromium will call VIDIOC_STREAMON twice with value of
sps->flags 0:
At the first time width and height are 16; ctx->coded_fmt.fmt.pix_mp.width
and coded_fmt.fmt.pix_mp.height are 16, which are the min size of decoder;
At the second time width and height are still 16; while
coded_fmt.fmt.pix_mp.width is 1920 and coded_fmt.fmt.pix_mp.height is
1088, which are the real size of video.
So VIDIOC_STREAMON will fall at the first time call because sps->flags is
0 so V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set, and then height is
doubled to 32 which is larger than 16.
What do you think if we skip doubling height if sps->flags is 0 and at the
same time V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set? The following hack
did fix my chromium:
--- a/drivers/staging/media/rkvdec2/rkvdec2-h264.c
+++ b/drivers/staging/media/rkvdec2/rkvdec2-h264.c
@@ -767,7 +767,7 @@ static int rkvdec2_h264_validate_sps(struct rkvdec2_ctx *ctx,
* which is half the final height (see (7-18) in the
* specification)
*/
- if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY))
+ if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY) && sps->flags)
height *= 2;
if (width > ctx->coded_fmt.fmt.pix_mp.width ||
Best regards
Jianfeng
