Patch "media: platform: mtk-mdp3: fix potential frame size overflow in mdp_try_fmt_mplane()" has been added to the 6.2-stable tree

[Sasha Levin <sashal@xxxxxxxxxx>]

This is a note to let you know that I've just added the patch titled

    media: platform: mtk-mdp3: fix potential frame size overflow in mdp_try_fmt_mplane()

to the 6.2-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
     media-platform-mtk-mdp3-fix-potential-frame-size-ove.patch
and it can be found in the queue-6.2 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@xxxxxxxxxxxxxxx> know about it.



commit 8681b0f0397d4ae5118e361e95490bbebd14957e
Author: Moudy Ho <moudy.ho@xxxxxxxxxxxx>
Date:   Mon Mar 27 11:13:24 2023 +0800

    media: platform: mtk-mdp3: fix potential frame size overflow in mdp_try_fmt_mplane()
    
    [ Upstream commit 4168720753ce6c14c5d3a35302fc2e1841383443 ]
    
    Fix overflow risk when setting certain formats whose frame size exceeds
    a RGB24 with 7723x7723 resolution.
    
    For example, a 7723x7724 RGB24 frame:
        1. bpl (byte per line) = 7723 * 3.
        2. Overflow occurs when bpl * 7724 * depth.
    
    Fixes: 61890ccaefaf ("media: platform: mtk-mdp3: add MediaTek MDP3 driver")
    Signed-off-by: Moudy Ho <moudy.ho@xxxxxxxxxxxx>
    Signed-off-by: Hans Verkuil <hverkuil-cisco@xxxxxxxxx>
    Signed-off-by: Sasha Levin <sashal@xxxxxxxxxx>

diff --git a/drivers/media/platform/mediatek/mdp3/mtk-mdp3-regs.c b/drivers/media/platform/mediatek/mdp3/mtk-mdp3-regs.c
index 4e84a37ecdfc1..36336d169bd91 100644
--- a/drivers/media/platform/mediatek/mdp3/mtk-mdp3-regs.c
+++ b/drivers/media/platform/mediatek/mdp3/mtk-mdp3-regs.c
@@ -4,6 +4,7 @@
  * Author: Ping-Hsun Wu <ping-hsun.wu@xxxxxxxxxxxx>
  */
 
+#include <linux/math64.h>
 #include <media/v4l2-common.h>
 #include <media/videobuf2-v4l2.h>
 #include <media/videobuf2-dma-contig.h>
@@ -428,14 +429,15 @@ const struct mdp_format *mdp_try_fmt_mplane(struct v4l2_format *f,
 		u32 bpl = pix_mp->plane_fmt[i].bytesperline;
 		u32 min_si, max_si;
 		u32 si = pix_mp->plane_fmt[i].sizeimage;
+		u64 di;
 
 		bpl = clamp(bpl, min_bpl, max_bpl);
 		pix_mp->plane_fmt[i].bytesperline = bpl;
 
-		min_si = (bpl * pix_mp->height * fmt->depth[i]) /
-			 fmt->row_depth[i];
-		max_si = (bpl * s.max_height * fmt->depth[i]) /
-			 fmt->row_depth[i];
+		di = (u64)bpl * pix_mp->height * fmt->depth[i];
+		min_si = (u32)div_u64(di, fmt->row_depth[i]);
+		di = (u64)bpl * s.max_height * fmt->depth[i];
+		max_si = (u32)div_u64(di, fmt->row_depth[i]);
 
 		si = clamp(si, min_si, max_si);
 		pix_mp->plane_fmt[i].sizeimage = si;