On 3/22/24 18:50, Kanchan Joshi wrote:
This patchset is aimed at getting the feedback on a new io_uring interface that userspace can use to exchange meta buffer along with read/write. Two new opcodes for that: IORING_OP_READ_META and IORING_OP_WRITE_META. The leftover space in the SQE is used to send meta buffer pointer and its length. Patch #2 for this.
I do remember there were back and forth design discussions about that back when some other guy attempted to implement it, but have you tried to do it not as a separate opcode? It reads like all read/write opcodes might benefit from it, and it'd be unfortunate to then be adding IORING_OP_READ_META_FIXED and multiplicatively all other variants.
The interface is supported for block direct IO. Patch #4 for this. Other two are prep patches. It has been tried not to touch the hot read/write path, as much as possible. Performance for non-meta IO is same after the patches [2]. There is some code in the cold path (worker-based async) though. Moderately tested by modifying the fio [1] to use this interface (only for NVMe block devices) [1] https://github.com/OpenMPDK/fio/tree/feat/test-meta [2] without this: taskset -c 2,5 t/io_uring -b512 -d128 -c32 -s32 -p1 -F1 -B1 -n2 -r4 /dev/nvme0n1 /dev/nvme1n1 submitter=1, tid=2453, file=/dev/nvme1n1, node=-1 submitter=0, tid=2452, file=/dev/nvme0n1, node=-1 polled=1, fixedbufs=1, register_files=1, buffered=0, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128 IOPS=10.02M, BW=4.89GiB/s, IOS/call=31/31 IOPS=10.04M, BW=4.90GiB/s, IOS/call=31/31 With this: taskset -c 2,5 t/io_uring -b512 -d128 -c32 -s32 -p1 -F1 -B1 -n2 -r4 /dev/nvme0n1 /dev/nvme1n1 submitter=1, tid=2453, file=/dev/nvme1n1, node=-1 submitter=0, tid=2452, file=/dev/nvme0n1, node=-1 polled=1, fixedbufs=1, register_files=1, buffered=0, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128 IOPS=10.02M, BW=4.89GiB/s, IOS/call=31/31 IOPS=10.04M, BW=4.90GiB/s, IOS/call=31/31 Anuj Gupta (3): io_uring/rw: Get rid of flags field in struct io_rw io_uring/rw: support read/write with metadata block: modify bio_integrity_map_user to accept iov_iter as argument Kanchan Joshi (1): block: add support to pass the meta buffer block/bio-integrity.c | 27 ++++++--- block/fops.c | 9 +++ block/t10-pi.c | 6 ++ drivers/nvme/host/ioctl.c | 11 +++- include/linux/bio.h | 13 +++- include/linux/fs.h | 1 + include/uapi/linux/io_uring.h | 6 ++ io_uring/io_uring.c | 2 + io_uring/opdef.c | 29 +++++++++ io_uring/rw.c | 108 +++++++++++++++++++++++++++++----- io_uring/rw.h | 8 +++ 11 files changed, 193 insertions(+), 27 deletions(-) base-commit: 6f0974eccbf78baead1735722c4f1ee3eb9422cd
-- Pavel Begunkov
