On 1/8/25 1:54 PM, Trevor Gamblin wrote: > > On 2025-01-07 16:02, David Lechner wrote: >> On 1/7/25 2:21 PM, Trevor Gamblin wrote: >>> On 2025-01-04 07:30, Jonathan Cameron wrote: >>>> On Thu, 2 Jan 2025 13:19:19 -0500 >>>> Trevor Gamblin <tgamblin@xxxxxxxxxxxx> wrote: >>>> >>>>> On 2024-12-19 11:13, Jonathan Cameron wrote: >>>>>> On Tue, 17 Dec 2024 16:47:28 -0500 >>>>>> Trevor Gamblin <tgamblin@xxxxxxxxxxxx> wrote: >>>>>> ... >>> else >>> val_calc = val * scale + val2 * scale / MICRO; >>> >>> val_calc /= 2; >>> >>> return clamp_t(int, val_calc, S16_MIN, S16_MAX); >>> } >>> >>> This seems to match all of the expected outputs for the pre-simplification version in this patch series when I test it. If there are no issues with it, I'll send a v2. >> Probably not a big deal, but there is unhanded overflow when val is near S32_MAX >> or S32_MIN. > Should I handle that with an extra call to clamp_t()? It wouldn't hurt. val = clamp_t(int, val, S32_MIN / 8, S32_MAX / 8); before the rest of the math should do the trick since the max scale is 8.
