I hacked a quick PoC, based on the input from my earlier post [1].
Quick recap; For certain XDP programs, would it be possible to get rid
of the xdp_do_redirect() call (and the per-cpu write/read), and
instead perform the action directly from the BPF helper? If so, that
would potentially make the XDP_REDIRECT faster/less complex.
This PoC/RFC teach the verifier to detect when an XDP program is
structured such that all bpf_redirect_map() calls are tail calls. A
driver can then probe the BPF program, use the new
xdp_set_redirect_tailcall() function, and avoid the xdp_do_redirect()
call. This was Toke's suggestion, instead of my XDP_CONSUMED idea,
adding a new action.
To perform the xdp_do_redirect() tasks from the bpf_redirect_map()
helper, additional parameters are needed (XDP context, netdev, and XDP
program). They are passed via the bpf_redirect_into per-cpu structure
using the xdp_set_redirect_tailcall() function.
Note that the code is broken for programs mixing bpf_redirect() and
bpf_redirect_map()!
There are more details in the commits.
Is this a good idea? I have only measured for AF_XDP redirects, but
all XDP_REDIRECT targets should benefit. For AF_XDP the rxdrop
scenario went from 21.5 to 23.2 Mpps on my machine.
Next steps would be implement proper tail calls (suggested by John and
Alexei).
Getting input on my (horrible) verifier peephole hack, and a better
way to detect tail calls would be very welcome!
Disregard naming and style. I'm mostly interested what people think
about the concept, and if it's worth working on.
Next steps:
* Better tail call detection in the verifier, instead of the naive
peephole version in patch 1.
* Implement proper tail calls (constrained, indirect jump BPF
instruction).
Cheers,
Björn
[1] https://lore.kernel.org/bpf/CAJ+HfNidbgwtLinLQohwocUmoYyRcAG454ggGkCbseQPSA1cpw@xxxxxxxxxxxxxx/
Björn Töpel (2):
bpf, xdp: add naive bpf_redirect_map() tail call detection
i40e: avoid xdp_do_redirect() call when "redirect_tail_call" is set
drivers/net/ethernet/intel/i40e/i40e_txrx.c | 14 ++++-
drivers/net/ethernet/intel/i40e/i40e_xsk.c | 14 ++++-
include/linux/bpf_verifier.h | 2 +
include/linux/filter.h | 19 ++++++-
kernel/bpf/verifier.c | 53 +++++++++++++++++++
net/core/filter.c | 57 +++++++++++++++++++++
6 files changed, 154 insertions(+), 5 deletions(-)
base-commit: cb8e59cc87201af93dfbb6c3dccc8fcad72a09c2
--
2.25.1
