On Mon, Apr 10, 2017 at 10:00 AM, Andreas Klinger <ak@xxxxxxxxxxxxx> wrote: > While calculating the compensation of the humidity there are negative values > interpreted as unsigned because of unsigned variables used. These values as > well as the constants need to be casted to signed as indicated by the > documentation of the sensor. > > Changes in v2: > - cast also constants to signed; otherwise calculation is not correct > > I removed the Acked-By of Linus Walleij because of another change in the > calculation. Don't know how it should be treated. > > Signed-off-by: Andreas Klinger <ak@xxxxxxxxxxxxx> Reviewed-by: Matt Ranostay <matt.ranostay@xxxxxxxxxxxx> > --- > drivers/iio/pressure/bmp280-core.c | 11 ++++++----- > 1 file changed, 6 insertions(+), 5 deletions(-) > > diff --git a/drivers/iio/pressure/bmp280-core.c b/drivers/iio/pressure/bmp280-core.c > index 4d18826ac63c..d82b788374b6 100644 > --- a/drivers/iio/pressure/bmp280-core.c > +++ b/drivers/iio/pressure/bmp280-core.c > @@ -175,11 +175,12 @@ static u32 bmp280_compensate_humidity(struct bmp280_data *data, > } > H6 = sign_extend32(tmp, 7); > > - var = ((s32)data->t_fine) - 76800; > - var = ((((adc_humidity << 14) - (H4 << 20) - (H5 * var)) + 16384) >> 15) > - * (((((((var * H6) >> 10) * (((var * H3) >> 11) + 32768)) >> 10) > - + 2097152) * H2 + 8192) >> 14); > - var -= ((((var >> 15) * (var >> 15)) >> 7) * H1) >> 4; > + var = ((s32)data->t_fine) - (s32)76800; > + var = ((((adc_humidity << 14) - (H4 << 20) - (H5 * var)) > + + (s32)16384) >> 15) * (((((((var * H6) >> 10) > + * (((var * (s32)H3) >> 11) + (s32)32768)) >> 10) > + + (s32)2097152) * H2 + 8192) >> 14); > + var -= ((((var >> 15) * (var >> 15)) >> 7) * (s32)H1) >> 4; > > return var >> 12; > }; > -- > 2.1.4 > > > -- -- To unsubscribe from this list: send the line "unsubscribe linux-iio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
