This is a streamlined series with new way of doing uring-cmd, and connects nvme-passthrough (over char device /dev/ngX) to it. uring-cmd enables using io_uring for any arbitrary command (ioctl, fsctl etc.) exposed by the command provider (e.g. driver, fs etc.). To store the command inline within the sqe, Jens added an option to setup the ring with 128-byte SQEs.This gives 80 bytes of space (16 bytes at the end of the first sqe + 64 bytes in the second sqe). With inline command in sqe, the application avoids explicit allocation and, in the kernel, we avoid doing copy_from_user. Command-opcode, length etc. are stored in per-op fields of io_uring_sqe. Non-inline submission (when command is a user-space pointer rather than housed inside sqe) is also supported. io_uring sends this command down by newly introduced ->async_cmd() handler in file_operations. The handler does what is required to submit, and indicates queued completion.The infra has been added to process the completion when it arrives. Overall the patches wire up the following capabilities for this path: - async - fixed-buffer - plugging - bio-cache - sync and async polling. This scales well. 512b randread perf (KIOPS) comparing uring-passthru-over-char (/dev/ng0n1) to uring-over-block(/dev/nvme0n1) QD uring pt uring-poll pt-poll 8 538 589 831 902 64 967 1131 1351 1378 256 1043 1230 1376 1429 Testing/perf is done with this custom fio that turnes regular-io into passthru-io on supplying "uring_cmd=1" option. https://github.com/joshkan/fio/tree/big-sqe-pt.v1 Example command-line: fio -iodepth=256 -rw=randread -ioengine=io_uring -bs=512 -numjobs=1 -runtime=60 -group_reporting -iodepth_batch_submit=64 -iodepth_batch_complete_min=1 -iodepth_batch_complete_max=64 -fixedbufs=1 -hipri=1 -sqthread_poll=0 -filename=/dev/ng0n1 -name=io_uring_256 -uring_cmd=1 Anuj Gupta (3): io_uring: prep for fixed-buffer enabled uring-cmd nvme: enable passthrough with fixed-buffer nvme: enable non-inline passthru commands Jens Axboe (5): io_uring: add support for 128-byte SQEs fs: add file_operations->async_cmd() io_uring: add infra and support for IORING_OP_URING_CMD io_uring: plug for async bypass block: wire-up support for plugging Kanchan Joshi (5): nvme: wire-up support for async-passthru on char-device. io_uring: add support for uring_cmd with fixed-buffer block: factor out helper for bio allocation from cache nvme: enable bio-cache for fixed-buffer passthru io_uring: add support for non-inline uring-cmd Keith Busch (2): nvme: modify nvme_alloc_request to take an additional parameter nvme: allow user passthrough commands to poll Pankaj Raghav (2): io_uring: add polling support for uring-cmd nvme: wire-up polling for uring-passthru block/bio.c | 43 ++-- block/blk-map.c | 45 +++++ block/blk-mq.c | 93 ++++----- drivers/nvme/host/core.c | 21 +- drivers/nvme/host/ioctl.c | 336 +++++++++++++++++++++++++++----- drivers/nvme/host/multipath.c | 2 + drivers/nvme/host/nvme.h | 11 +- drivers/nvme/host/pci.c | 4 +- drivers/nvme/target/passthru.c | 2 +- fs/io_uring.c | 188 ++++++++++++++++-- include/linux/bio.h | 1 + include/linux/blk-mq.h | 4 + include/linux/fs.h | 2 + include/linux/io_uring.h | 43 ++++ include/uapi/linux/io_uring.h | 21 +- include/uapi/linux/nvme_ioctl.h | 4 + 16 files changed, 689 insertions(+), 131 deletions(-) -- 2.25.1
