On Wed, Jan 23, 2019 at 11:17 AM Phil Reid <preid@xxxxxxxxxxxxxxxxx> wrote: > > What's the best way to handle scale values for very high precision ADC's. > > eg a 24 or even a 32 bit dac with a 2.5v full scale range. > > Even using the nV conversion param it looses a fair bit of precision. > > Would a option to return a rational number would be better? Are you saying something like float/double C types ? Typically those are not allowed in kernel. Scale values are typically/mostly for informational purposes. i.e. the driver computes the scales [and the current selected scales] for the measured values, so that the userspace parts [usually libiio] can compute actual values So, you could compute & display scales almost any way you want it, even just displaying a string representation of the scale value. It is good to follow some conventions though. You could take a look at various formats for the scales. Something like IIO_VAL_INT_PLUS_NANO or IIO_VAL_FRACTIONAL or IIO_VAL_FRACTIONAL_LOG2 [see __iio_format_value in drivers/iio/industrialio-core.c to get some idea ] . Depends what you really need. I guess to give a better answer here [I deferred giving one hoping someone more informed would jump-in], we would also need a more concrete example. Thanks Alex > > Or am I doing something wrong. > > -- > Regards > Phil
