This is an RFC for the alternative approach suggested by Martin and Jakub. I've tried to CC most of the people from the original discussion, feel free to add more if you think I've missed somebody. Summary: - add new BPF_F_XDP_HAS_METADATA program flag and abuse attr->prog_ifindex to pass target device ifindex at load time - at load time, find appropriate ndo_unroll_kfunc and call it to unroll/inline kfuncs; kfuncs have the default "safe" implementation if unrolling is not supported by a particular device - rewrite xskxceiver test to use C bpf program and extend it to export rx_timestamp (plus add rx timestamp to veth driver) I've intentionally kept it small and hacky to see whether the approach is workable or not. Pros: - we avoid BTF complexity; the BPF programs themselves are now responsible for agreeing on the metadata layout with the AF_XDP consumer - the metadata is free if not used - the metadata should, in theory, be cheap if used; kfuncs should be unrolled to the same code as if the metadata was pre-populated and passed with a BTF id - it's not all or nothing; users can use small subset of metadata which is more efficient than the BTF id approach where all metadata has to be exposed for every frame (and selectively consumed by the users) Cons: - forwarding has to be handled explicitly; the BPF programs have to agree on the metadata layout (IOW, the forwarding program has to be aware of the final AF_XDP consumer metadata layout) - TX picture is not clear; but it's not clear with BTF ids as well; I think we've agreed that just reusing whatever we have at RX won't fly at TX; seems like TX XDP program might be the answer here? (with a set of another tx kfuncs to "expose" bpf/af_xdp metatata back into the kernel) Cc: Martin KaFai Lau <martin.lau@xxxxxxxxx> Cc: Jakub Kicinski <kuba@xxxxxxxxxx> Cc: Willem de Bruijn <willemb@xxxxxxxxxx> Cc: Jesper Dangaard Brouer <brouer@xxxxxxxxxx> Cc: Anatoly Burakov <anatoly.burakov@xxxxxxxxx> Cc: Alexander Lobakin <alexandr.lobakin@xxxxxxxxx> Cc: Magnus Karlsson <magnus.karlsson@xxxxxxxxx> Cc: Maryam Tahhan <mtahhan@xxxxxxxxxx> Cc: xdp-hints@xxxxxxxxxxxxxxx Cc: netdev@xxxxxxxxxxxxxxx Stanislav Fomichev (5): bpf: Support inlined/unrolled kfuncs for xdp metadata veth: Support rx timestamp metadata for xdp libbpf: Pass prog_ifindex via bpf_object_open_opts selftests/bpf: Convert xskxceiver to use custom program selftests/bpf: Test rx_timestamp metadata in xskxceiver drivers/net/veth.c | 31 +++++ include/linux/bpf.h | 1 + include/linux/btf.h | 1 + include/linux/btf_ids.h | 4 + include/linux/netdevice.h | 3 + include/net/xdp.h | 22 ++++ include/uapi/linux/bpf.h | 5 + kernel/bpf/syscall.c | 28 ++++- kernel/bpf/verifier.c | 60 +++++++++ net/core/dev.c | 7 ++ net/core/xdp.c | 28 +++++ tools/include/uapi/linux/bpf.h | 5 + tools/lib/bpf/libbpf.c | 1 + tools/lib/bpf/libbpf.h | 6 +- tools/testing/selftests/bpf/Makefile | 1 + .../testing/selftests/bpf/progs/xskxceiver.c | 43 +++++++ tools/testing/selftests/bpf/xskxceiver.c | 119 +++++++++++++++--- tools/testing/selftests/bpf/xskxceiver.h | 5 +- 18 files changed, 348 insertions(+), 22 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/xskxceiver.c -- 2.38.1.273.g43a17bfeac-goog
