The frmsize structure was left initialize to 0, as side effect, the
driver was reporting an invalid frmsize.
Size: Stepwise 0x0 - 0x0 with step 0/0
The frmsize structure in hantro_fmt is only valid for bitstream formats.
So just adding frmsizes for non coded formats does not work to fix this.
When codec_mode is HANTRO_MODE_NONE, then vidioc_enum_framesizes should
return with -ENOTTY. At least when hantro_needs_postproc returns false.
Which is the case for encoders. But with the latest postproc scaling
patch this is not the case anymore. This patch is fixing this back to
how it was but changing the EINVAL to ENOTTY since we touch this anyway.
For now this is a good enough fix, but in the future enum_framesizes
needs to obtain different constraints in framesizes in combination of
coded to raw formats.
Fixes: 79c987de8b35 ("media: hantro: Use post processor scaling capacities")
Signed-off-by: Michael Grzeschik <m.grzeschik@xxxxxxxxxxxxxx>
Reviewed-by: Ezequiel Garcia <ezequiel@xxxxxxxxxxxxxxxxxxxx>
---
v1 -> v2: - changed EINVAL to ENOTTY as suggested by Nicolas
- updated the commit message to add more infos to the issue
drivers/media/platform/verisilicon/hantro_v4l2.c | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c
index 2c7a805289e7b8..30e650edaea8a3 100644
--- a/drivers/media/platform/verisilicon/hantro_v4l2.c
+++ b/drivers/media/platform/verisilicon/hantro_v4l2.c
@@ -161,8 +161,11 @@ static int vidioc_enum_framesizes(struct file *file, void *priv,
}
/* For non-coded formats check if postprocessing scaling is possible */
- if (fmt->codec_mode == HANTRO_MODE_NONE && hantro_needs_postproc(ctx, fmt)) {
- return hanto_postproc_enum_framesizes(ctx, fsize);
+ if (fmt->codec_mode == HANTRO_MODE_NONE) {
+ if (hantro_needs_postproc(ctx, fmt))
+ return hanto_postproc_enum_framesizes(ctx, fsize);
+ else
+ return -ENOTTY;
} else if (fsize->index != 0) {
vpu_debug(0, "invalid frame size index (expected 0, got %d)\n",
fsize->index);
--
2.30.2
