If we have io-wq or SQPOLL setting the task_work notify signal but the
task itself doesn't have task_work to process, we don't clear the
flag and hence will enter a repeated check loop if we're waiting on
events or file/buf references to go away.
This was introduced in a recent patch which eliminated gating the
task_work run on just that flag, but that fix meant that we know don't
clear the flag if the task itsel doesn't have task_work to run.
Cc: stable@xxxxxxxxxxxxxxx
Fixes: 46a525e199e4 ("io_uring: don't gate task_work run on TIF_NOTIFY_SIGNAL")
Signed-off-by: Jens Axboe <axboe@xxxxxxxxx>
---
V2
- Add comment
- Keep the clear-if-set logic, just move it outside the tw check
diff --git a/io_uring/io_uring.h b/io_uring/io_uring.h
index cef5ff924e63..a70f8ec88bf3 100644
--- a/io_uring/io_uring.h
+++ b/io_uring/io_uring.h
@@ -238,9 +238,16 @@ static inline unsigned int io_sqring_entries(struct io_ring_ctx *ctx)
static inline int io_run_task_work(void)
{
+ /*
+ * Always check-and-clear the task_work notification signal. The
+ * current task may have it set if io-wq or SQPOLL sets it, but not
+ * have any task_work itself. This can prevent the current task from
+ * waiting on events efficiently, as interruptible sleeps will turn
+ * into busy loops.
+ */
+ if (test_thread_flag(TIF_NOTIFY_SIGNAL))
+ clear_notify_signal();
if (task_work_pending(current)) {
- if (test_thread_flag(TIF_NOTIFY_SIGNAL))
- clear_notify_signal();
__set_current_state(TASK_RUNNING);
task_work_run();
return 1;
--
Jens Axboe
