Hi, Currently uring-cmd lacks the ability to leverage the pre-registered buffers. This series adds the support in uring-cmd, and plumbs nvme passthrough to work with it. Using registered-buffers showed peak-perf hike from 1.85M to 2.17M IOPS in my setup. Without fixedbufs ***************** # taskset -c 0 t/io_uring -b512 -d128 -c32 -s32 -p0 -F1 -B0 -O0 -n1 -u1 /dev/ng0n1 submitter=0, tid=5256, file=/dev/ng0n1, node=-1 polled=0, fixedbufs=0/0, register_files=1, buffered=1, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128 IOPS=1.85M, BW=904MiB/s, IOS/call=32/31 IOPS=1.85M, BW=903MiB/s, IOS/call=32/32 IOPS=1.85M, BW=902MiB/s, IOS/call=32/32 ^CExiting on signal Maximum IOPS=1.85M With fixedbufs ************** # taskset -c 0 t/io_uring -b512 -d128 -c32 -s32 -p0 -F1 -B1 -O0 -n1 -u1 /dev/ng0n1 submitter=0, tid=5260, file=/dev/ng0n1, node=-1 polled=0, fixedbufs=1/0, register_files=1, buffered=1, QD=128 Engine=io_uring, sq_ring=128, cq_ring=128 IOPS=2.17M, BW=1059MiB/s, IOS/call=32/31 IOPS=2.17M, BW=1057MiB/s, IOS/call=32/32 IOPS=2.16M, BW=1055MiB/s, IOS/call=32/32 ^CExiting on signal Maximum IOPS=2.17M Patch 1, 3 = prep/infrastructure Patch 2 = expand io_uring command to use registered-buffers Patch 4 = expand nvme passthrough to use registered-buffers This series is prepared on top of: for-next + iopoll-passthru series [1]. A unified branch is present here: https://github.com/OpenMPDK/linux/commits/feat/pt_fixedbufs_v3 t/io_uring util with fixedbuf support is here: https://github.com/joshkan/fio/tree/priv/fb-v3 Changes since v2: - Kill the new opcode, add a flag instead (Pavel) - Fix standalone build issue with patch 1 (Pavel) Changes since v1: - Fix a naming issue for an exported helper [1] https://lore.kernel.org/io-uring/20220823161443.49436-1-joshi.k@xxxxxxxxxxx/ Anuj Gupta (2): io_uring: introduce io_uring_cmd_import_fixed io_uring: introduce fixed buffer support for io_uring_cmd Kanchan Joshi (2): block: add helper to map bvec iterator for passthrough nvme: wire up fixed buffer support for nvme passthrough block/blk-map.c | 71 +++++++++++++++++++++++++++++++++++ drivers/nvme/host/ioctl.c | 38 +++++++++++++------ include/linux/blk-mq.h | 1 + include/linux/io_uring.h | 11 +++++- include/uapi/linux/io_uring.h | 9 +++++ io_uring/uring_cmd.c | 29 +++++++++++++- 6 files changed, 145 insertions(+), 14 deletions(-) -- 2.25.1
