Hi! On 2022-01-09 14:02, Andy Shevchenko wrote: > On Sat, Jan 8, 2022 at 10:53 PM Liam Beguin <liambeguin@xxxxxxxxx> wrote: >> >> The approximation caused by integer divisions can be costly on smaller >> scale values since the decimal part is significant compared to the >> integer part. Switch to an IIO_VAL_INT_PLUS_NANO scale type in such >> cases to maintain accuracy. > ... > ... > >> + rem2 = *val % (int)tmp; >> + *val = *val / (int)tmp; >> + >> + *val2 = rem / (int)tmp; > > Hmm... You divide s64 by 10^9, which means that the maximum value can > be ~10^10 / 2 (because 2^64-1 ~= 10^19), but this _might_ be bigger > than 'int' can hold. Can you confirm that tmp can't be so big? > If tmp is so big that it does not fit in 32-bits, that is indeed a problem, and it means that the scale has overflowed. However, the problem with the scale overflowing very much existed prior to this series. Doing something about that overflow is not strictly related to improving the accuracy for small scale factors. Cheers, Peter
