WARNING: it's an early prototype and could likely be broken and unsafe
to run. Also, most probably it doesn't do the right thing from the
modern BPF perspective, but that's fine as I want to get some numbers
first and only then consult with BPF folks and brush it up.
A comeback of the io_uring BPF proposal put on top new infrastructure.
Instead executing BPF as a new request type, it's now run in the io_uring
waiting loop. The program is called to react every time we get a new
event like a queued task_work or an interrupt. Patch 3 adds some helpers
the BPF program can use to interact with io_uring like submitting new
requests and looking at CQEs. It also controls when to return control
back to user space by returning one of IOU_BPF_RET_{OK,STOP}, and sets
the task_work batching size, i.e. how many CQEs to wait for it be run
again, via a kfunc helper. We need to be able to sleep to submit
requests, hence only sleepable BPF is allowed.
BPF can help to create arbitrary relations between requests from
within the kernel and later help with tuning the wait loop batching.
E.g. with minor extensions we can implement batch wait timeouts.
We can also use it to let the user to safely access internal resources
and maybe even do a more elaborate request setup than SQE allows it.
The benchmark is primitive, the non-BPF baseline issues a 2 nop request
link at a time and waits for them to complete. The BPF version runs
them (2 * N requests) one by one. Numbers with mitigations on:
# nice -n -20 taskset -c 0 ./minimal 0 50000000
type 2-LINK, requests to run 50000000
sec 10, total (ms) 10314
# nice -n -20 taskset -c 0 ./minimal 1 50000000
type BPF, requests to run 50000000
sec 6, total (ms) 6808
It needs to be better tested, especially with asynchronous requests
like reads and other hardware. It can also be further optimised. E.g.
we can avoid extra locking by taking it once for BPF/task_work_run.
The test (see examples-bpf/minimal[.bpf].c)
https://github.com/isilence/liburing.git io_uring-bpf
https://github.com/isilence/liburing/tree/io_uring-bpf
Pavel Begunkov (3):
bpf/io_uring: add io_uring program type
io_uring/bpf: allow to register and run BPF programs
io_uring/bpf: add kfuncs for BPF programs
include/linux/bpf.h | 1 +
include/linux/bpf_types.h | 4 +
include/linux/io_uring/bpf.h | 10 ++
include/linux/io_uring_types.h | 4 +
include/uapi/linux/bpf.h | 1 +
include/uapi/linux/io_uring.h | 9 ++
include/uapi/linux/io_uring/bpf.h | 22 ++++
io_uring/Makefile | 1 +
io_uring/bpf.c | 205 ++++++++++++++++++++++++++++++
io_uring/bpf.h | 43 +++++++
io_uring/io_uring.c | 16 +++
io_uring/register.c | 7 +
kernel/bpf/btf.c | 3 +
kernel/bpf/syscall.c | 1 +
kernel/bpf/verifier.c | 10 +-
15 files changed, 336 insertions(+), 1 deletion(-)
create mode 100644 include/linux/io_uring/bpf.h
create mode 100644 include/uapi/linux/io_uring/bpf.h
create mode 100644 io_uring/bpf.c
create mode 100644 io_uring/bpf.h
--
2.46.0
