read_poll_timeout_atomic() uses ktime_get() to implement the timeout
feature, just like its non-atomic counterpart. However, there are
several issues with this, due to its use in atomic contexts:
1. When called in the s2ram path (as typically done by clock or PM
domain drivers), timekeeping may be suspended, triggering the
WARN_ON(timekeeping_suspended) in ktime_get():
WARNING: CPU: 0 PID: 654 at kernel/time/timekeeping.c:843 ktime_get+0x28/0x78
Calling ktime_get_mono_fast_ns() instead of ktime_get() would get
rid of that warning. However, that would break timeout handling,
as (at least on systems with an ARM architectured timer), the time
returned by ktime_get_mono_fast_ns() does not advance while
timekeeping is suspended.
Interestingly, (on the same ARM systems) the time returned by
ktime_get() does advance while timekeeping is suspended, despite
the warning.
2. Depending on the actual clock source, and especially before a
high-resolution clocksource (e.g. the ARM architectured timer)
becomes available, time may not advance in atomic contexts, thus
breaking timeout handling.
Fix this by abandoning the idea that one can rely on timekeeping to
implement timeout handling in all atomic contexts, and switch from a
global time-based to a locally-estimated timeout handling. In most
(all?) cases the timeout condition is exceptional and an error
condition, hence any additional delays due to underestimating wall clock
time are irrelevant.
Signed-off-by: Geert Uytterhoeven <geert+renesas@xxxxxxxxx>
---
Alternatively, one could use a mixed approach (use both
ktime_get_mono_fast_ns() and a local (under)estimate, and timeout on the
earliest occasion), but I think that would complicate things without
much gain.
v2:
- New.
---
include/linux/iopoll.h | 18 +++++++++++++-----
1 file changed, 13 insertions(+), 5 deletions(-)
diff --git a/include/linux/iopoll.h b/include/linux/iopoll.h
index 0417360a6db9b0d6..bb2e1d9117e96679 100644
--- a/include/linux/iopoll.h
+++ b/include/linux/iopoll.h
@@ -81,22 +81,30 @@
delay_before_read, args...) \
({ \
u64 __timeout_us = (timeout_us); \
+ s64 __left_ns = __timeout_us * NSEC_PER_USEC; \
unsigned long __delay_us = (delay_us); \
- ktime_t __timeout = ktime_add_us(ktime_get(), __timeout_us); \
- if (delay_before_read && __delay_us) \
+ u64 __delay_ns = __delay_us * NSEC_PER_USEC; \
+ if (delay_before_read && __delay_us) { \
udelay(__delay_us); \
+ if (__timeout_us) \
+ __left_ns -= __delay_ns; \
+ } \
for (;;) { \
(val) = op(args); \
if (cond) \
break; \
- if (__timeout_us && \
- ktime_compare(ktime_get(), __timeout) > 0) { \
+ if (__timeout_us && __left_ns < 0) { \
(val) = op(args); \
break; \
} \
- if (__delay_us) \
+ if (__delay_us) { \
udelay(__delay_us); \
+ if (__timeout_us) \
+ __left_ns -= __delay_ns; \
+ } \
cpu_relax(); \
+ if (__timeout_us) \
+ __left_ns--; \
} \
(cond) ? 0 : -ETIMEDOUT; \
})
--
2.34.1
