Computation of sens2 was wrong and is fixed by this patch, sens2 should be:
2 * (t - 2000)^2
See page 8 of ms5607 datasheet here:
http://www.meas-spec.com/product/pressure/MS5607-02BA03.aspx
Signed-off-by: Gregor Boirie <gregor.boirie@xxxxxxxxxx>
Acked-by: Lucas De Marchi <lucas.demarchi@xxxxxxxxx>
Acked-by: Peter Meerwald-Stadler <pmeerw@xxxxxxxxxx>
---
drivers/iio/pressure/ms5611_core.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/iio/pressure/ms5611_core.c b/drivers/iio/pressure/ms5611_core.c
index c7885f0c..84ab8d2 100644
--- a/drivers/iio/pressure/ms5611_core.c
+++ b/drivers/iio/pressure/ms5611_core.c
@@ -136,17 +136,17 @@ static int ms5607_temp_and_pressure_compensate(struct ms5611_chip_info *chip_inf
t = 2000 + ((chip_info->prom[6] * dt) >> 23);
if (t < 2000) {
- s64 off2, sens2, t2;
+ s64 off2, sens2, t2, tmp;
t2 = (dt * dt) >> 31;
- off2 = (61 * (t - 2000) * (t - 2000)) >> 4;
- sens2 = off2 << 1;
+ tmp = (t - 2000) * (t - 2000);
+ off2 = (61 * tmp) >> 4;
+ sens2 = tmp << 1;
if (t < -1500) {
- s64 tmp = (t + 1500) * (t + 1500);
-
+ tmp = (t + 1500) * (t + 1500);
off2 += 15 * tmp;
- sens2 += (8 * tmp);
+ sens2 += 8 * tmp;
}
t -= t2;
--
2.1.4
--
To unsubscribe from this list: send the line "unsubscribe linux-iio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
