This RFC patchset adds asynchronous ioctl capability for NVMe devices. Purpose of RFC is to get the feedback and optimize the path. At the uppermost io-uring layer, a new opcode IORING_OP_IOCTL_PT is presented to user-space applications. Like regular-ioctl, it takes ioctl opcode and an optional argument (ioctl-specific input/output parameter). Unlike regular-ioctl, it is made to skip the block-layer and reach directly to the underlying driver (nvme in the case of this patchset). This path between io-uring and nvme is via a newly introduced block-device operation "async_ioctl". This operation expects io-uring to supply a callback function which can be used to report completion at later stage. For a regular ioctl, NVMe driver submits the command to the device and the submitter (task) is made to wait until completion arrives. For async-ioctl, completion is decoupled from submission. Submitter goes back to its business without waiting for nvme-completion. When nvme-completion arrives, it informs io-uring via the registered completion-handler. But some ioctls may require updating certain ioctl-specific fields which can be accessed only in context of the submitter task. For that reason, NVMe driver uses task-work infra for that ioctl-specific update. Since task-work is not exported, it cannot be referenced when nvme is compiled as a module. Therefore, one of the patch exports task-work API. Here goes example of usage (pseudo-code). Actual nvme-cli source, modified to issue all ioctls via this opcode is present at- https://github.com/joshkan/nvme-cli/commit/a008a733f24ab5593e7874cfbc69ee04e88068c5 With regular ioctl- int nvme_submit_passthru(int fd, unsigned long ioctl_cmd, struct nvme_passthru_cmd *cmd) { return ioctl(fd, ioctl_cmd, cmd); } With uring passthru ioctl- int nvme_submit_passthru(int fd, unsigned long ioctl_cmd, struct nvme_passthru_cmd *cmd) { return uring_ioctl(fd, ioctl_cmd, cmd); } int uring_ioctl(int fd, unsinged long cmd, u64 arg) { sqe = io_uring_get_sqe(ring); /* prepare sqe */ sqe->fd = fd; sqe->opcode = IORING_OP_IOCTL_PT; sqe->ioctl_cmd = cmd; sqe->ioctl_arg = arg; /* submit sqe */ io_uring_submit(ring); /* reap completion and obtain result */ io_uring_wait_cqe(ring, &cqe); printf("ioctl result =%d\n", cqe->res) } Kanchan Joshi (4): block: introduce async ioctl operation kernel: export task_work_add nvme: add async ioctl support io_uring: add async passthrough ioctl support drivers/nvme/host/core.c | 347 +++++++++++++++++++++++++++------- fs/io_uring.c | 77 ++++++++ include/linux/blkdev.h | 12 ++ include/uapi/linux/io_uring.h | 7 +- kernel/task_work.c | 2 +- 5 files changed, 376 insertions(+), 69 deletions(-) -- 2.25.1
