Re: [PATCH v11 08/15] iio: afe: rescale: fix accuracy for small fractional scales

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Dec 22, 2021 at 5:47 AM Liam Beguin <liambeguin@xxxxxxxxx> wrote:
>
> From: Liam Beguin <lvb@xxxxxxxxxx>
>
> The approximation caused by integer divisions can be costly on smaller
> scale values since the decimal part is significant compared to the
> integer part. Switch to an IIO_VAL_INT_PLUS_NANO scale type in such
> cases to maintain accuracy.

...

> -               tmp = div_s64(tmp, 1000000000LL);
> +
> +               tmp = div_s64_rem(tmp, 1000000000LL, &rem);
>                 *val = tmp;

> -               return scale_type;
> +

It seems you may add this blank line to one of the previous patches, may you?

> +               if (!rem)
> +                       return scale_type;

-- 
With Best Regards,
Andy Shevchenko



[Index of Archives]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Input]     [Linux Kernel]     [Linux SCSI]     [X.org]

  Powered by Linux