Hi Linus,
Here are the io_uring changes slated for the 6.13 kernel release. This
pull request contains:
- Cleanups of the eventfd handling code, making it fully private.
- Support for sending a sync message to another ring, without having a
ring available to send a normal async message.
- Get rid of the separate unlocked hash table, unify everything around
the single locked one.
- Add support for ring resizing. It can be hard to appropriately size
the CQ ring upfront, if the application doesn't know how busy it will
be. This results in applications sizing rings for the most busy case,
which can be wasteful. With ring resizing, they can start small and
grow the ring, if needed.
- Add support for fixed wait regions, rather than needing to copy the
same wait data tons of times for each wait operation.
- Rewrite the resource node handling, which before was serialized per
ring. This caused issues with particularly fixed files, where one file
waiting on IO could hold up putting and freeing of other unrelated
files. Now each node is handled separately. New code is much simpler
too, and was a net 250 line reduction in code.
- Add support for just doing partial buffer clones, rather than always
cloning the entire buffer table.
- Series adding static NAPI support, where a specific NAPI instance is
used rather than having a list of them available that need lookup.
- Add support for mapped regions, and also convert the fixed wait
support mentioned above to that concept. This avoids doing special
mappings for various planned features, and folds the existing
registered wait into that too.
- Add support for hybrid IO polling, which is a variant of strict IOPOLL
but with an initial sleep delay to avoid spinning too early and
wasting resources on devices that aren't necessarily in the < 5 usec
category wrt latencies.
- Various cleanups and little fixes.
Please pull!
The following changes since commit 42f7652d3eb527d03665b09edac47f85fb600924:
Linux 6.12-rc4 (2024-10-20 15:19:38 -0700)
are available in the Git repository at:
git://git.kernel.dk/linux.git tags/for-6.13/io_uring-20241118
for you to fetch changes up to a652958888fb1ada3e4f6b548576c2d2c1b60d66:
io_uring/region: fix error codes after failed vmap (2024-11-17 09:01:35 -0700)
----------------------------------------------------------------
for-6.13/io_uring-20241118
----------------------------------------------------------------
Al Viro (1):
switch io_msg_ring() to CLASS(fd)
Jens Axboe (50):
io_uring/eventfd: abstract out ev_fd put helper
io_uring/eventfd: check for the need to async notifier earlier
io_uring/eventfd: move actual signaling part into separate helper
io_uring/eventfd: move trigger check into a helper
io_uring/eventfd: abstract out ev_fd grab + release helpers
io_uring/eventfd: move ctx->evfd_last_cq_tail into io_ev_fd
io_uring/msg_ring: refactor a few helper functions
io_uring/msg_ring: add support for sending a sync message
io_uring/poll: remove 'ctx' argument from io_poll_req_delete()
io_uring/poll: get rid of unlocked cancel hash
io_uring/poll: get rid of io_poll_tw_hash_eject()
io_uring/poll: get rid of per-hashtable bucket locks
io_uring/cancel: get rid of init_hash_table() helper
io_uring: move cancel hash tables to kvmalloc/kvfree
io_uring/rsrc: don't assign bvec twice in io_import_fixed()
io_uring/uring_cmd: get rid of using req->imu
io_uring/rw: get rid of using req->imu
io_uring: remove 'issue_flags' argument for io_req_set_rsrc_node()
io_uring/net: move send zc fixed buffer import to issue path
io_uring: kill 'imu' from struct io_kiocb
io_uring: move max entry definition and ring sizing into header
io_uring: abstract out a bit of the ring filling logic
io_uring/memmap: explicitly return -EFAULT for mmap on NULL rings
io_uring/register: add IORING_REGISTER_RESIZE_RINGS
io_uring/sqpoll: wait on sqd->wait for thread parking
io_uring: switch struct ext_arg from __kernel_timespec to timespec64
io_uring: change io_get_ext_arg() to use uaccess begin + end
io_uring: add support for fixed wait regions
io_uring/nop: add support for testing registered files and buffers
io_uring/rsrc: move struct io_fixed_file to rsrc.h header
io_uring: specify freeptr usage for SLAB_TYPESAFE_BY_RCU io_kiocb cache
io_uring/splice: open code 2nd direct file assignment
io_uring/rsrc: kill io_charge_rsrc_node()
io_uring/rsrc: get rid of per-ring io_rsrc_node list
io_uring/rsrc: get rid of io_rsrc_node allocation cache
io_uring/rsrc: add an empty io_rsrc_node for sparse buffer entries
io_uring: only initialize io_kiocb rsrc_nodes when needed
io_uring/rsrc: unify file and buffer resource tables
io_uring/rsrc: add io_rsrc_node_lookup() helper
io_uring/filetable: remove io_file_from_index() helper
io_uring/filetable: kill io_reset_alloc_hint() helper
io_uring/rsrc: add io_reset_rsrc_node() helper
io_uring/rsrc: get rid of the empty node and dummy_ubuf
io_uring/rsrc: allow cloning at an offset
io_uring/rsrc: allow cloning with node replacements
io_uring/rsrc: encode node type and ctx together
io_uring/rsrc: split io_kiocb node type assignments
io_uring: move cancelations to be io_uring_task based
io_uring: remove task ref helpers
io_uring: move struct io_kiocb from task_struct to io_uring_task
Ming Lei (4):
io_uring/rsrc: pass 'struct io_ring_ctx' reference to rsrc helpers
io_uring/rsrc: remove '->ctx_ptr' of 'struct io_rsrc_node'
io_uring/rsrc: add & apply io_req_assign_buf_node()
io_uring/uring_cmd: fix buffer index retrieval
Olivier Langlois (6):
io_uring/napi: protect concurrent io_napi_entry timeout accesses
io_uring/napi: fix io_napi_entry RCU accesses
io_uring/napi: improve __io_napi_add
io_uring/napi: Use lock guards
io_uring/napi: clean up __io_napi_do_busy_loop
io_uring/napi: add static napi tracking strategy
Pavel Begunkov (17):
io_uring: kill io_llist_xchg
io_uring: static_key for !IORING_SETUP_NO_SQARRAY
io_uring: clean up cqe trace points
io_uring/net: split send and sendmsg prep helpers
io_uring/net: don't store send address ptr
io_uring/net: don't alias send user pointer reads
io_uring/net: clean up io_msg_copy_hdr
io_uring: prevent speculating sq_array indexing
io_uring: avoid normal tw intermediate fallback
io_uring: fix invalid hybrid polling ctx leaks
io_uring: fortify io_pin_pages with a warning
io_uring: disable ENTER_EXT_ARG_REG for IOPOLL
io_uring: temporarily disable registered waits
io_uring: introduce concept of memory regions
io_uring: add memory region registration
io_uring: restore back registered wait arguments
io_uring/region: fix error codes after failed vmap
hexue (1):
io_uring: add support for hybrid IOPOLL
include/linux/io_uring/cmd.h | 2 +-
include/linux/io_uring_types.h | 88 ++++--
include/trace/events/io_uring.h | 24 +-
include/uapi/linux/io_uring.h | 119 +++++++-
io_uring/cancel.c | 20 +-
io_uring/cancel.h | 1 -
io_uring/eventfd.c | 137 ++++++---
io_uring/fdinfo.c | 88 ++++--
io_uring/filetable.c | 71 ++---
io_uring/filetable.h | 35 +--
io_uring/futex.c | 4 +-
io_uring/futex.h | 4 +-
io_uring/io_uring.c | 433 ++++++++++++++------------
io_uring/io_uring.h | 30 +-
io_uring/memmap.c | 83 +++++
io_uring/memmap.h | 14 +
io_uring/msg_ring.c | 91 ++++--
io_uring/msg_ring.h | 1 +
io_uring/napi.c | 184 +++++++----
io_uring/napi.h | 8 +-
io_uring/net.c | 112 ++++---
io_uring/nop.c | 47 ++-
io_uring/notif.c | 7 +-
io_uring/opdef.c | 2 +
io_uring/poll.c | 181 +++--------
io_uring/poll.h | 2 +-
io_uring/register.c | 299 +++++++++++++++++-
io_uring/rsrc.c | 657 +++++++++++++++-------------------------
io_uring/rsrc.h | 97 +++---
io_uring/rw.c | 105 +++++--
io_uring/splice.c | 42 ++-
io_uring/splice.h | 1 +
io_uring/sqpoll.c | 3 +-
io_uring/tctx.c | 1 +
io_uring/timeout.c | 16 +-
io_uring/timeout.h | 2 +-
io_uring/uring_cmd.c | 27 +-
io_uring/uring_cmd.h | 2 +-
io_uring/waitid.c | 6 +-
io_uring/waitid.h | 2 +-
40 files changed, 1825 insertions(+), 1223 deletions(-)
--
Jens Axboe
