This is a note to let you know that I've just added the patch titled io_uring: cmpxchg for poll arm refs release to the 6.0-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary The filename of the patch is: io_uring-cmpxchg-for-poll-arm-refs-release.patch and it can be found in the queue-6.0 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable@xxxxxxxxxxxxxxx> know about it. >From 2f3893437a4ebf2e892ca172e9e122841319d675 Mon Sep 17 00:00:00 2001 From: Pavel Begunkov <asml.silence@xxxxxxxxx> Date: Sun, 20 Nov 2022 16:57:41 +0000 Subject: io_uring: cmpxchg for poll arm refs release From: Pavel Begunkov <asml.silence@xxxxxxxxx> commit 2f3893437a4ebf2e892ca172e9e122841319d675 upstream. Replace atomically substracting the ownership reference at the end of arming a poll with a cmpxchg. We try to release ownership by setting 0 assuming that poll_refs didn't change while we were arming. If it did change, we keep the ownership and use it to queue a tw, which is fully capable to process all events and (even tolerates spurious wake ups). It's a bit more elegant as we reduce races b/w setting the cancellation flag and getting refs with this release, and with that we don't have to worry about any kinds of underflows. It's not the fastest path for polling. The performance difference b/w cmpxchg and atomic dec is usually negligible and it's not the fastest path. Cc: stable@xxxxxxxxxxxxxxx Fixes: aa43477b04025 ("io_uring: poll rework") Signed-off-by: Pavel Begunkov <asml.silence@xxxxxxxxx> Link: https://lore.kernel.org/r/0c95251624397ea6def568ff040cad2d7926fd51.1668963050.git.asml.silence@xxxxxxxxx Signed-off-by: Jens Axboe <axboe@xxxxxxxxx> Signed-off-by: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx> --- io_uring/poll.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) --- a/io_uring/poll.c +++ b/io_uring/poll.c @@ -519,7 +519,6 @@ static int __io_arm_poll_handler(struct unsigned issue_flags) { struct io_ring_ctx *ctx = req->ctx; - int v; INIT_HLIST_NODE(&req->hash_node); req->work.cancel_seq = atomic_read(&ctx->cancel_seq); @@ -587,11 +586,10 @@ static int __io_arm_poll_handler(struct if (ipt->owning) { /* - * Release ownership. If someone tried to queue a tw while it was - * locked, kick it off for them. + * Try to release ownership. If we see a change of state, e.g. + * poll was waken up, queue up a tw, it'll deal with it. */ - v = atomic_dec_return(&req->poll_refs); - if (unlikely(v & IO_POLL_REF_MASK)) + if (atomic_cmpxchg(&req->poll_refs, 1, 0) != 1) __io_poll_execute(req, 0); } return 0; Patches currently in stable-queue which might be from asml.silence@xxxxxxxxx are queue-6.0/selftests-net-don-t-tests-batched-tcp-io_uring-zc.patch queue-6.0/io_uring-make-poll-refs-more-robust.patch queue-6.0/io_uring-cmpxchg-for-poll-arm-refs-release.patch queue-6.0/io_uring-poll-fix-poll_refs-race-with-cancelation.patch queue-6.0/io_uring-poll-lockdep-annote-io_poll_req_insert_lock.patch