On 2021/08/30 22:47, Dan Carpenter wrote:
> On Mon, Aug 30, 2021 at 10:37:31PM +0900, Tetsuo Handa wrote:
>> On 2021/08/30 22:00, Dan Carpenter wrote:
>>>>> diff --git a/drivers/video/fbdev/vga16fb.c b/drivers/video/fbdev/vga16fb.c
>>>>> index e2757ff1c23d..e483a3f5fd47 100644
>>>>> --- a/drivers/video/fbdev/vga16fb.c
>>>>> +++ b/drivers/video/fbdev/vga16fb.c
>>>>> @@ -403,7 +403,7 @@ static int vga16fb_check_var(struct fb_var_screeninfo *var,
>>>>>
>>>>> if (yres > vyres)
>>>>> vyres = yres;
>>>>> - if (vxres * vyres > maxmem) {
>>>>> + if ((u64) vxres * vyres > (u64) maxmem) {
>>>>
>>>> Mindlessly changing the sizes is not the solution.
>>>> Please use e.g. the array_size() helper from <linux/overflow.h>
>>>> instead.
>>>
>>> On a 64bit system the array_size() macro is going to do the exact same
>>> casts? But I do think this code would be easier to understand if the
>>> integer overflow check were pull out separately and done first:
>>>
>>> if (array_size(vxres, vyres) >= UINT_MAX)
>>> return -EINVAL;
>>
>> This is wrong. array_size() returns ULONG_MAX on 64bits upon overflow and
>> returns UINT_MAX on 32bits upon overflow. However, UINT_MAX is a valid
>> value without overflow (e.g. vxres == UINT_MAX / 15 && vyres == 15).
>
> Huh... I just assumed we didn't allow resolutions that high.
Of course, we don't allow resolutions that high. ;-)
Since I don't know possible max resolutions, I chose UINT_MAX + 1 as a common
limit for returning -EINVAL. Unless overflow happens, vga16fb_check_var() will
return -ENOMEM on such high resolutions.
>
>> Comparing like "> (u64) UINT_MAX" is to detect only overflow.
>>
>
> Of course, that doesn't work on 32 bit systems. Also the cast isn't
> required because of type promotion.
Indeed, "> UINT_MAX" seems to work on both 32bits and 64bits.
----------
#include <stdio.h>
#include <limits.h>
int main(int argc, char *argv[])
{
unsigned int w = 0x600;
unsigned int h = 0x10000000;
if ((unsigned long long) w * h > UINT_MAX)
printf("Overflowed\n");
else
printf("No overflow\n");
return 0;
}
----------
