Hi, This patchset gets rid of on-stack state, that is then fixed up and copied if we need to go async. Having to do this fixup is nasty business, and this is the main motivation for the change. Opcodes are converted to setting up their async context at prep time, which means that everything is stable beyond that. No more special io_req_prep_async() handling, and no more "oops we can't proceed, let's now allocate memory, copy state, and be ready for a retry". By default, opcodes are now always ready for a retry, and the issue path can be simplified. This is most readily apparent in the read/write handling, but can be seen on the net side too. Lastly, the alloc cache is rewritten to be array based rather than list based. List based isn't a great choice, as grabbing an element from the list also means you have to touch the next one. With all of that, performance is as good as before, or better, and we drop quite a bit of code. The diffstat reflects that, but doesn't even tell the full story. Most of the added lines are trivial, whereas some of the removed lines are pretty hairy. Changes since v1: - Cleanups - Switch connect to using io_async_msghdr, now it gets recycling too - Avoid recycling for read/write if io-wq is used - Fix errant io_async_rw shadowing in io_write() - Change alloc_cache to be array based - Fix KASAN issues. Not with mem reuse, but just errors in my implementation of it for the mempool. - Only mark iovec caching as REQ_F_NEED_CLEANUP - Shuffle some hunks around between patches - Fix an issue with send zerocopy and iovec freeing - Move connect to io_async_msghdr so it can tap into the recycling - Actually delete struct io_rw_state, not just its elements - Add uring_cmd optimization that avoids sqe copy unless needed - Rebase on for-6.10/io_uring include/linux/io_uring_types.h | 4 +- io_uring/alloc_cache.h | 51 ++-- io_uring/futex.c | 26 +- io_uring/futex.h | 5 +- io_uring/io_uring.c | 71 ++--- io_uring/io_uring.h | 1 - io_uring/net.c | 550 +++++++++++++++++---------------------- io_uring/net.h | 27 +- io_uring/opdef.c | 65 ++--- io_uring/opdef.h | 9 +- io_uring/poll.c | 11 +- io_uring/poll.h | 7 +- io_uring/rsrc.c | 9 +- io_uring/rsrc.h | 5 +- io_uring/rw.c | 570 +++++++++++++++++++++-------------------- io_uring/rw.h | 25 +- io_uring/uring_cmd.c | 75 ++++-- io_uring/uring_cmd.h | 7 +- 18 files changed, 707 insertions(+), 811 deletions(-) -- Jens Axboe
