This series adds support for executing multiple XDP programs on a single interface in sequence, through the use of chain calls, as discussed at the Linux Plumbers Conference last month: https://linuxplumbersconf.org/event/4/contributions/460/ # HIGH-LEVEL IDEA Since the response to the previous iteration was pretty unanimous that this should not be XDP-specific, this version takes a different approach: We add the ability to inject chain call programs into the eBPF execution core itself. This also turns out to be simpler, so that's good :) The way this new approach works is the bpf() syscall gets a couple of new commands which takes a pair of BPF program fds and a return code. It will then attach the second program to the first one in a structured keyed by return code. When a program chain is thus established, the former program will tail call to the latter at the end of its execution. The actual tail calling is achieved by having the verifier inject instructions into the program that performs the chain call lookup and tail call before each BPF_EXIT instruction. Since this rewriting has to be performed at program load time, a new flag has to be set to trigger the rewriting. Only programs loaded with this flag set can have other programs attached to them for chain calls. Ideally, it shouldn't be necessary to set the flag on program load time, but rather inject the calls when a chain call program is first loaded. However, rewriting the program reallocates the bpf_prog struct, which is obviously not possible after the program has been attached to something. One way around this could be a sysctl to force the flag one (for enforcing system-wide support). Another could be to have the chain call support itself built into the interpreter and JIT, which could conceivably be re-run each time we attach a new chain call program. This would also allow the JIT to inject direct calls to the next program instead of using the tail call infrastructure, which presumably would be a performance win. The drawback is, of course, that it would require modifying all the JITs. # PERFORMANCE I performed a simple performance test to get an initial feel for the overhead of the chain call mechanism. This test consists of running only two programs in sequence: One that returns XDP_PASS and another that returns XDP_DROP. I then measure the drop PPS performance and compare it to a baseline of just a single program that only returns XDP_DROP. For comparison, a test case that uses regular eBPF tail calls to sequence two programs together is also included. I did not re-run the baseline tests from before, so the two top values are the same | Test case | Perf | Overhead | |----------------------------------+-----------+----------| | Before patch (XDP DROP program) | 31.0 Mpps | | | XDP tail call | 26.6 Mpps | 5.3 ns | |----------------------------------+-----------+----------| | After patch (XDP DROP program) | 31.2 Mpps | | | XDP chain call (wildcard return) | 26.8 Mpps | 5.3 ns | | XDP chain call (XDP_PASS return) | 27.2 Mpps | 4.7 ns | The difference between the wildcard and XDP_PASS cases for chain calls, is that using the wildcard mode needs two tail call attempts where the first one fails, (see the injected BPF code in patch 1) while XDP_PASS matches on the first tail call. # PATCH SET STRUCTURE This series is structured as follows: - Patch 1: Adds the code that injects the instructions into the programs - Patch 2: Adds the new commands added to the bpf() syscall - Patch 3-4: Tools/ update and libbpf syscall wrappers - Patch 5: Selftest with example user space code (a bit hacky still) The whole series is also available in my git repo on kernel.org: https://git.kernel.org/pub/scm/linux/kernel/git/toke/linux.git/log/?h=xdp-multiprog-02 Changelog: v2: - Completely new approach that integrates chain calls into the core eBPF runtime instead of doing the map XDP-specific thing with a new map from v1. --- Alan Maguire (1): bpf: Add support for setting chain call sequence for programs Toke Høiland-Jørgensen (4): bpf: Support injecting chain calls into BPF programs on load tools: Update bpf.h header for program chain calls libbpf: Add syscall wrappers for BPF_PROG_CHAIN_* commands selftests: Add tests for XDP chain calls include/linux/bpf.h | 2 include/uapi/linux/bpf.h | 16 + kernel/bpf/core.c | 10 + kernel/bpf/syscall.c | 81 ++++++ kernel/bpf/verifier.c | 76 ++++++ tools/include/uapi/linux/bpf.h | 16 + tools/lib/bpf/bpf.c | 34 +++ tools/lib/bpf/bpf.h | 4 tools/lib/bpf/libbpf.map | 3 tools/testing/selftests/bpf/.gitignore | 1 tools/testing/selftests/bpf/Makefile | 3 tools/testing/selftests/bpf/progs/xdp_dummy.c | 6 tools/testing/selftests/bpf/test_xdp_chain.sh | 77 ++++++ tools/testing/selftests/bpf/xdp_chain.c | 313 +++++++++++++++++++++++++ 14 files changed, 640 insertions(+), 2 deletions(-) create mode 100755 tools/testing/selftests/bpf/test_xdp_chain.sh create mode 100644 tools/testing/selftests/bpf/xdp_chain.c
