On 2021/08/30 22:00, Dan Carpenter wrote: >>> diff --git a/drivers/video/fbdev/vga16fb.c b/drivers/video/fbdev/vga16fb.c >>> index e2757ff1c23d..e483a3f5fd47 100644 >>> --- a/drivers/video/fbdev/vga16fb.c >>> +++ b/drivers/video/fbdev/vga16fb.c >>> @@ -403,7 +403,7 @@ static int vga16fb_check_var(struct fb_var_screeninfo *var, >>> >>> if (yres > vyres) >>> vyres = yres; >>> - if (vxres * vyres > maxmem) { >>> + if ((u64) vxres * vyres > (u64) maxmem) { >> >> Mindlessly changing the sizes is not the solution. >> Please use e.g. the array_size() helper from <linux/overflow.h> >> instead. > > On a 64bit system the array_size() macro is going to do the exact same > casts? But I do think this code would be easier to understand if the > integer overflow check were pull out separately and done first: > > if (array_size(vxres, vyres) >= UINT_MAX) > return -EINVAL; This is wrong. array_size() returns ULONG_MAX on 64bits upon overflow and returns UINT_MAX on 32bits upon overflow. However, UINT_MAX is a valid value without overflow (e.g. vxres == UINT_MAX / 15 && vyres == 15). Comparing like "> (u64) UINT_MAX" is to detect only overflow. array_size() would be helpful for forcing memory allocation to fail (instead of allocating smaller than actually required). > > if (vxres * vyres > maxmem) { > ... > > The UINT_MAX is because vxres and vyres are u32. > > This would maybe be the first time anyone ever did an integer overflow > check like this in the kernel. It's a new idiom. > > regards, > dan carpenter >