Hi Linus,
This is the set of io_uring updates and fixes for the 6.4 merge window.
Nothing major in this one, details below:
- Cleanup of the io-wq per-node mapping, notably getting rid of it so we
just have a single io_wq entry per ring (Breno)
- Followup to the above, move accounting to io_wq as well and completely
drop struct io_wqe (Gabriel)
- Enable KASAN for the internal io_uring caches (Breno)
- Add support for multishot timeouts. Some applications use timeouts to
wake someone waiting on completion entries, and this makes it a bit
easier to just have a recurring timer rather than needing to rearm it
every time (David)
- Support archs that have shared cache coloring between userspace and
the kernel, and hence have strict address requirements for mmap'ing
the ring into userspace. This should only be parisc/hppa. (Helge, me)
- XFS has supported O_DIRECT writes without needing to lock the inode
exclusively for a long time, and ext4 now supports it as well. This is
true for the common cases of not extending the file size. Flag the fs
as having that feature, and utilize that to avoid serializing those
writes in io_uring (me)
- Enable completion batching for uring commands (me)
- Revert patch adding io_uring restriction to what can be GUP mapped or
not. This does not belong in io_uring, as io_uring isn't really
special in this regard. Since this is also getting in the way of
cleanups and improvements to the GUP code, get rid of if (me)
- A few series greatly reducing the complexity of registered resources,
like buffers or files. Not only does this clean up the code a lot, the
simplified code is also a LOT more efficient (Pavel)
- Series optimizing how we wait for events and run task_work related to
it (Pavel)
- Fixes for file/buffer unregistration with DEFER_TASKRUN (Pavel)
- Misc cleanups and improvements (Pavel, me)
Please pull for 6.4-rc1!
The following changes since commit 7e364e56293bb98cae1b55fd835f5991c4e96e7d:
Linux 6.3-rc5 (2023-04-02 14:29:29 -0700)
are available in the Git repository at:
git://git.kernel.dk/linux.git tags/for-6.4/io_uring-2023-04-21
for you to fetch changes up to 3c85cc43c8e7855d202da184baf00c7b8eeacf71:
Revert "io_uring/rsrc: disallow multi-source reg buffers" (2023-04-20 06:51:48 -0600)
----------------------------------------------------------------
for-6.4/io_uring-2023-04-21
----------------------------------------------------------------
Breno Leitao (3):
io_uring: One wqe per wq
io_uring: Move from hlist to io_wq_work_node
io_uring: Add KASAN support for alloc_caches
David Wei (1):
io_uring: add support for multishot timeouts
Gabriel Krisman Bertazi (2):
io-wq: Move wq accounting to io_wq
io-wq: Drop struct io_wqe
Helge Deller (1):
io_uring: Adjust mapping wrt architecture aliasing requirements
Jens Axboe (13):
fs: add FMODE_DIO_PARALLEL_WRITE flag
io_uring: avoid hashing O_DIRECT writes if the filesystem doesn't need it
io_uring/kbuf: move pinning of provided buffer ring into helper
io_uring/kbuf: add buffer_list->is_mapped member
io_uring/kbuf: rename struct io_uring_buf_reg 'pad' to'flags'
io_uring: add support for user mapped provided buffer ring
io_uring/kbuf: disallow mapping a badly aligned provided ring buffer
io_uring/io-wq: drop outdated comment
io_uring: rename trace_io_uring_submit_sqe() tracepoint
io_uring: cap io_sqring_entries() at SQ ring size
io_uring/uring_cmd: assign ioucmd->cmd at async prep time
io_uring/uring_cmd: take advantage of completion batching
Revert "io_uring/rsrc: disallow multi-source reg buffers"
Pavel Begunkov (51):
io_uring: kill unused notif declarations
io_uring: remove extra tw trylocks
io_uring: encapsulate task_work state
io_uring/rsrc: use non-pcpu refcounts for nodes
io_uring/rsrc: keep cached refs per node
io_uring: don't put nodes under spinlocks
io_uring: io_free_req() via tw
io_uring/rsrc: protect node refs with uring_lock
io_uring/rsrc: kill rsrc_ref_lock
io_uring/rsrc: rename rsrc_list
io_uring/rsrc: optimise io_rsrc_put allocation
io_uring/rsrc: don't offload node free
io_uring/rsrc: cache struct io_rsrc_node
io_uring/rsrc: add lockdep sanity checks
io_uring/rsrc: optimise io_rsrc_data refcounting
io_uring/rsrc: add custom limit for node caching
io_uring: move pinning out of io_req_local_work_add
io_uring: optimize local tw add ctx pinning
io_uring: refactor io_cqring_wake()
io_uring: add tw add flags
io_uring: inline llist_add()
io_uring: reduce scheduling due to tw
io_uring: refactor __io_cq_unlock_post_flush()
io_uring: optimise io_req_local_work_add
io_uring: shut io_prep_async_work warning
io_uring/kbuf: remove extra ->buf_ring null check
io_uring: add irq lockdep checks
io_uring/rsrc: add lockdep checks
io_uring/rsrc: consolidate node caching
io_uring/rsrc: zero node's rsrc data on alloc
io_uring/rsrc: refactor io_rsrc_node_switch
io_uring/rsrc: extract SCM file put helper
io_uring/notif: add constant for ubuf_info flags
io_uring/rsrc: use nospec'ed indexes
io_uring/rsrc: remove io_rsrc_node::done
io_uring/rsrc: refactor io_rsrc_ref_quiesce
io_uring/rsrc: use wq for quiescing
io_uring/rsrc: fix DEFER_TASKRUN rsrc quiesce
io_uring/rsrc: remove rsrc_data refs
io_uring/rsrc: inline switch_start fast path
io_uring/rsrc: clean up __io_sqe_buffers_update()
io_uring/rsrc: simplify single file node switching
io_uring/rsrc: refactor io_queue_rsrc_removal
io_uring/rsrc: remove unused io_rsrc_node::llist
io_uring/rsrc: infer node from ctx on io_queue_rsrc_removal
io_uring/rsrc: merge nodes and io_rsrc_put
io_uring/rsrc: add empty flag in rsrc_node
io_uring/rsrc: inline io_rsrc_put_work()
io_uring/rsrc: pass node to io_rsrc_put_work()
io_uring/rsrc: devirtualise rsrc put callbacks
io_uring/rsrc: disassociate nodes and rsrc_data
fs/ext4/file.c | 3 +-
fs/xfs/xfs_file.c | 3 +-
include/linux/fs.h | 3 +
include/linux/io_uring_types.h | 24 +-
include/trace/events/io_uring.h | 15 +-
include/uapi/linux/io_uring.h | 33 +--
io_uring/alloc_cache.h | 39 ++-
io_uring/filetable.c | 21 +-
io_uring/io-wq.c | 524 +++++++++++++++++-----------------------
io_uring/io_uring.c | 348 +++++++++++++++++---------
io_uring/io_uring.h | 49 ++--
io_uring/kbuf.c | 160 +++++++++---
io_uring/kbuf.h | 7 +
io_uring/net.h | 5 +-
io_uring/notif.c | 8 +-
io_uring/notif.h | 3 +-
io_uring/poll.c | 32 +--
io_uring/rsrc.c | 350 +++++++++------------------
io_uring/rsrc.h | 72 +++---
io_uring/rw.c | 8 +-
io_uring/timeout.c | 71 +++++-
io_uring/uring_cmd.c | 18 +-
22 files changed, 949 insertions(+), 847 deletions(-)
--
Jens Axboe
