What branch is this against? Do you have a git tree available? On Tue, Mar 08, 2022 at 08:50:48PM +0530, Kanchan Joshi wrote: > This is a streamlined series with new way of doing uring-cmd, and connects > nvme-passthrough (over char device /dev/ngX) to it. > uring-cmd enables using io_uring for any arbitrary command (ioctl, > fsctl etc.) exposed by the command provider (e.g. driver, fs etc.). > > To store the command inline within the sqe, Jens added an option to setup > the ring with 128-byte SQEs.This gives 80 bytes of space (16 bytes at > the end of the first sqe + 64 bytes in the second sqe). With inline > command in sqe, the application avoids explicit allocation and, in the > kernel, we avoid doing copy_from_user. Command-opcode, length etc. > are stored in per-op fields of io_uring_sqe. > > Non-inline submission (when command is a user-space pointer rather than > housed inside sqe) is also supported. > > io_uring sends this command down by newly introduced ->async_cmd() > handler in file_operations. The handler does what is required to > submit, and indicates queued completion.The infra has been added to > process the completion when it arrives. > > Overall the patches wire up the following capabilities for this path: > - async > - fixed-buffer > - plugging > - bio-cache > - sync and async polling. > > This scales well. 512b randread perf (KIOPS) comparing > uring-passthru-over-char (/dev/ng0n1) to > uring-over-block(/dev/nvme0n1) > > QD uring pt uring-poll pt-poll > 8 538 589 831 902 > 64 967 1131 1351 1378 > 256 1043 1230 1376 1429 > > Testing/perf is done with this custom fio that turnes regular-io into > passthru-io on supplying "uring_cmd=1" option. > https://github.com/joshkan/fio/tree/big-sqe-pt.v1 > > Example command-line: > fio -iodepth=256 -rw=randread -ioengine=io_uring -bs=512 -numjobs=1 > -runtime=60 -group_reporting -iodepth_batch_submit=64 > -iodepth_batch_complete_min=1 -iodepth_batch_complete_max=64 > -fixedbufs=1 -hipri=1 -sqthread_poll=0 -filename=/dev/ng0n1 > -name=io_uring_256 -uring_cmd=1 > > > Anuj Gupta (3): > io_uring: prep for fixed-buffer enabled uring-cmd > nvme: enable passthrough with fixed-buffer > nvme: enable non-inline passthru commands > > Jens Axboe (5): > io_uring: add support for 128-byte SQEs > fs: add file_operations->async_cmd() > io_uring: add infra and support for IORING_OP_URING_CMD > io_uring: plug for async bypass > block: wire-up support for plugging > > Kanchan Joshi (5): > nvme: wire-up support for async-passthru on char-device. > io_uring: add support for uring_cmd with fixed-buffer > block: factor out helper for bio allocation from cache > nvme: enable bio-cache for fixed-buffer passthru > io_uring: add support for non-inline uring-cmd > > Keith Busch (2): > nvme: modify nvme_alloc_request to take an additional parameter > nvme: allow user passthrough commands to poll > > Pankaj Raghav (2): > io_uring: add polling support for uring-cmd > nvme: wire-up polling for uring-passthru > > block/bio.c | 43 ++-- > block/blk-map.c | 45 +++++ > block/blk-mq.c | 93 ++++----- > drivers/nvme/host/core.c | 21 +- > drivers/nvme/host/ioctl.c | 336 +++++++++++++++++++++++++++----- > drivers/nvme/host/multipath.c | 2 + > drivers/nvme/host/nvme.h | 11 +- > drivers/nvme/host/pci.c | 4 +- > drivers/nvme/target/passthru.c | 2 +- > fs/io_uring.c | 188 ++++++++++++++++-- > include/linux/bio.h | 1 + > include/linux/blk-mq.h | 4 + > include/linux/fs.h | 2 + > include/linux/io_uring.h | 43 ++++ > include/uapi/linux/io_uring.h | 21 +- > include/uapi/linux/nvme_ioctl.h | 4 + > 16 files changed, 689 insertions(+), 131 deletions(-) > > -- > 2.25.1 ---end quoted text---
