Re: [PATCH v6 09/13] iio: afe: rescale: fix precision on fractional log scale

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 2021-07-21 05:06, Liam Beguin wrote:
> From: Liam Beguin <lvb@xxxxxxxxxx>
> 
> The IIO_VAL_FRACTIONAL_LOG2 scale type doesn't return the expected
> scale. Update the case so that the rescaler returns a fractional type
> and a more precise scale.
> 
> Signed-off-by: Liam Beguin <lvb@xxxxxxxxxx>
> ---
>  drivers/iio/afe/iio-rescale.c | 9 +++------
>  1 file changed, 3 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/iio/afe/iio-rescale.c b/drivers/iio/afe/iio-rescale.c
> index 35fa3b4e53e0..47cd4a6d9aca 100644
> --- a/drivers/iio/afe/iio-rescale.c
> +++ b/drivers/iio/afe/iio-rescale.c
> @@ -44,12 +44,9 @@ int rescale_process_scale(struct rescale *rescale, int scale_type,
>  		*val2 = rescale->denominator;
>  		return IIO_VAL_FRACTIONAL;
>  	case IIO_VAL_FRACTIONAL_LOG2:
> -		tmp = *val * 1000000000LL;
> -		do_div(tmp, rescale->denominator);
> -		tmp *= rescale->numerator;
> -		do_div(tmp, 1000000000LL);
> -		*val = tmp;
> -		return scale_type;
> +		*val = rescale->numerator * *val;
> +		*val2 = rescale->denominator * (1 << *val2);
> +		return IIO_VAL_FRACTIONAL;

Hi!

I do not think this is an uncontested improvement. You have broken the case
where *val2 is "large" before the scale factor is applied.

Cheers,
Peter

>  	case IIO_VAL_INT_PLUS_NANO:
>  		tmp = ((s64)*val * 1000000000LL + *val2) * rescale->numerator;
>  		tmp = div_s64(tmp, rescale->denominator);
> 



[Index of Archives]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Input]     [Linux Kernel]     [Linux SCSI]     [X.org]

  Powered by Linux