In mx2_camera_try_fmt(), when applying i.MX25 restrictions to frame sizes,
the height is checked to be <= 0xffff. But later an integer multiplication
height * 4 * 0x3ffff is performed, which will overflow even for bounded
height values. This patch switches to using 64-bit multiplication and
division to avoid overflowing.
Signed-off-by: Guennadi Liakhovetski <g.liakhovetski@xxxxxx>
---
Any objections anyone?
drivers/media/video/mx2_camera.c | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/media/video/mx2_camera.c b/drivers/media/video/mx2_camera.c
index 18afaee..ae1420b 100644
--- a/drivers/media/video/mx2_camera.c
+++ b/drivers/media/video/mx2_camera.c
@@ -22,6 +22,7 @@
#include <linux/gcd.h>
#include <linux/interrupt.h>
#include <linux/kernel.h>
+#include <linux/math64.h>
#include <linux/mm.h>
#include <linux/moduleparam.h>
#include <linux/time.h>
@@ -1367,8 +1368,8 @@ static int mx2_camera_try_fmt(struct soc_camera_device *icd,
/* Check against the CSIRXCNT limit */
if (pix->sizeimage > 4 * 0x3ffff) {
/* Adjust geometry, preserve aspect ratio */
- unsigned int new_height = int_sqrt(4 * 0x3ffff *
- pix->height / pix->bytesperline);
+ unsigned int new_height = int_sqrt(div_u64(0x3ffffULL *
+ 4 * pix->height, pix->bytesperline));
pix->width = new_height * pix->width / pix->height;
pix->height = new_height;
pix->bytesperline = soc_mbus_bytes_per_line(pix->width,
--
1.7.2.5
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
