Agreed. We may be slightly at cross-purposes here.
Sticking with a divisor of 256, I make the nearest consistent set of
scale factors to be as follows:
static int asc7621_in_scaling[] = {
3333, 3000, 4400, 6667, 16000
};
These would do pretty well and as you say, the rounding errors would be
far too small to cause a problem. I suppose its a matter of taste and
sticking with the data sheet that makes me prefer the 3/4 scale ratios.
My apologies for what has turned into a slightly pedantic digression!
Not sure why I called this thread "Temperature scaling" either... ;-)
Regards,
Ken.
On 07/03/10 12:10, Jean Delvare wrote:
Hi Ken,
On Sun, 07 Mar 2010 11:30:13 +0000, Ken Milmore wrote:
Example:
The 5V sensor with an input of precisely 5V should give an input MSB
register value of 192.
Using the "full scale" ratios:
192 * 6640 / 256 = 4980
Using the 3/4 scale ratios:
192 * 5000 / 192 = 5000
Error calculation:
(5000 - 4980) / 5000 = 0.4%
This corresponds to a 1-bit error in the 8 bit voltages, or a 3-bit
error in the 10 bit voltages, effectively giving only 7-bit accuracy in
the calculated values. An unnecessary error I think you'll agree.
Although the full scale values could probably be "tweaked" to make them
slightly better, the rounding issues inherent with integer arithmetic
make it difficult to get right on the button near the nominal voltages,
where highest accuracy is appropriate. This is easily avoided by using
the simple 3/4 scale ratios given in the data sheet.
No, you're off track, sorry. The only problem in the driver's formula is
that 6640 was computed as 5000 * 255 / 192, but the formula divides by
256 instead of 255. This is the inconsistency that leads to the 0.4%
error (1/256 = 0.4%). Just use consistent numbers and it will work
fine. The rounding issues inherent to integer arithmetic induce much
smaller errors, down to a level you won't notice them. In the case
above, the max error would be 0.5/6640 = 0.0075%... Way below the
resolution and accuracy of the sensors.
Now, if you insist on using the 3/4 scale values as a basis, really, I
don't care. All I'm saying is this this design change is not _required_
to make the driver return correct values.
_______________________________________________
lm-sensors mailing list
lm-sensors@xxxxxxxxxxxxxx
http://lists.lm-sensors.org/mailman/listinfo/lm-sensors
