This series extends bind() for XDP sockets, so that the bound socket
is added to the netdev_rx_queue _rx array in the netdevice. We call
this to register the socket. To redirect packets to the registered
socket, a new BPF helper is used: bpf_redirect_xsk().
For shared XDP sockets, only the first bound socket is
registered. Users that need more complex setup has to use XSKMAP and
bpf_redirect_map().
Now, why would one use bpf_redirect_xsk() over the regular
bpf_redirect_map() helper?
* Better performance!
* Convenience; Most user use one socket per queue. This scenario is
what registered sockets support. There is no need to create an
XSKMAP. This can also reduce complexity from containerized setups,
where users might what to use XDP sockets without CAP_SYS_ADMIN
capabilities.
The first patch restructures xdp_do_redirect() a bit, to make it
easier to add the new helper. This restructure also give us a slight
performance benefit. The following three patches extends bind() and
adds the new helper. After that, two libbpf patches that selects XDP
program based on what kernel is running. Finally, selftests for the new
functionality is added.
Note that the libbpf "auto-selection" is based on kernel version, so
it is hard coded to the "-next" version (5.12). If you would like to
try this is out, you will need to change the libbpf patch locally!
Thanks to Maciej and Magnus for the internal review/comments!
Performance (rxdrop, zero-copy)
Baseline
Two cores: 21.3 Mpps
One core: 24.5 Mpps
Patched
Two cores, bpf_redirect_map: 21.7 Mpps + 2%
One core, bpf_redirect_map: 24.9 Mpps + 2%
Two cores, bpf_redirect_xsk: 24.0 Mpps +13%
One core, bpf_redirect_xsk: 25.5 Mpps + 4%
Thanks!
Björn
v1->v2:
* Added missing XDP programs to selftests.
* Fixed checkpatch warning in selftests.
Björn Töpel (8):
xdp: restructure redirect actions
xsk: remove explicit_free parameter from __xsk_rcv()
xsk: fold xp_assign_dev and __xp_assign_dev
xsk: register XDP sockets at bind(), and add new AF_XDP BPF helper
libbpf, xsk: select AF_XDP BPF program based on kernel version
libbpf, xsk: select bpf_redirect_xsk(), if supported
selftest/bpf: add XDP socket tests for bpf_redirect_{xsk, map}()
selftest/bpf: remove a lot of ifobject casting in xdpxceiver
include/linux/filter.h | 10 +
include/linux/netdevice.h | 1 +
include/net/xdp_sock.h | 12 +
include/net/xsk_buff_pool.h | 2 +-
include/trace/events/xdp.h | 46 ++--
include/uapi/linux/bpf.h | 7 +
net/core/filter.c | 205 ++++++++++--------
net/xdp/xsk.c | 112 ++++++++--
net/xdp/xsk_buff_pool.c | 12 +-
tools/include/uapi/linux/bpf.h | 7 +
tools/lib/bpf/libbpf.c | 2 +-
tools/lib/bpf/libbpf_internal.h | 2 +
tools/lib/bpf/libbpf_probes.c | 16 --
tools/lib/bpf/xsk.c | 83 ++++++-
.../selftests/bpf/progs/xdpxceiver_ext1.c | 15 ++
.../selftests/bpf/progs/xdpxceiver_ext2.c | 9 +
tools/testing/selftests/bpf/test_xsk.sh | 48 ++++
tools/testing/selftests/bpf/xdpxceiver.c | 164 +++++++++-----
tools/testing/selftests/bpf/xdpxceiver.h | 2 +
19 files changed, 554 insertions(+), 201 deletions(-)
create mode 100644 tools/testing/selftests/bpf/progs/xdpxceiver_ext1.c
create mode 100644 tools/testing/selftests/bpf/progs/xdpxceiver_ext2.c
base-commit: 95204c9bfa48d2f4d3bab7df55c1cc823957ff81
--
2.27.0
