On 10/08/2018 09:08 PM, Hans Verkuil wrote:
> When the OSD is on (i.e. vivid displays text on top of the test pattern), and
> you enable hflip, then the driver crashes.
>
> The cause turned out to be a division of a negative number by an unsigned value.
> You expect that -8 / 2 would be -4, but in reality it is 2147483644 :-(
>
> Signed-off-by: Hans Verkuil <hans.verkuil@xxxxxxxxx>
> Reported-by: Mauro Carvalho Chehab <mchehab+samsung@xxxxxxxxxx>
> ---
> diff --git a/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c b/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c
> index f3d9c1140ffa..e76f87dc4368 100644
> --- a/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c
> +++ b/drivers/media/common/v4l2-tpg/v4l2-tpg-core.c
> @@ -1773,7 +1773,7 @@ typedef struct { u16 __; u8 _; } __packed x24;
> pos[7] = (chr & (0x01 << 0) ? fg : bg); \
> } \
> \
> - pos += (tpg->hflip ? -8 : 8) / hdiv; \
> + pos += (tpg->hflip ? -8 : 8) / (int)hdiv; \
> } \
> } \
> } while (0)
>
This can be CC-ed to stable for 4.7 and up.
It actually broke in 4.1, but it was called vivid-tpg.c at that time.
Regards,
Hans
