On Wed, Apr 14, 2021 at 1:02 PM Andrii Nakryiko <andrii@xxxxxxxxxx> wrote: > > Add BPF static linker support for extern resolution of global variables, > functions, and BTF-defined maps. > > This patch set consists of 4 parts: > - few patches are extending bpftool to simplify working with BTF dump; > - libbpf object loading logic is extended to support __hidden functions and > overriden (unused) __weak functions; also BTF-defined map parsing logic is > refactored to be re-used by linker; > - the crux of the patch set is BPF static linker logic extension to perform > extern resolution for three categories: global variables, BPF > sub-programs, BTF-defined maps; > - a set of selftests that validate that all the combinations of > extern/weak/__hidden are working as expected. > > See respective patches for more details. > > One aspect hasn't been addressed yet and is going to be resolved in the next > patch set, but is worth mentioning. With BPF static linking of multiple .o > files, dealing with static everything becomes more problematic for BPF > skeleton and in general for any by name look up APIs. This is due to static > entities are allowed to have non-unique name. Historically this was never > a problem due to BPF programs were always confined to a single C file. That > changes now and needs to be addressed. The thinking so far is for BPF static > linker to prepend filename to each static variable and static map (which is > currently not supported by libbpf, btw), so that they can be unambiguously > resolved by (mostly) unique name. Mostly, because even filenames can be > duplicated, but that should be rare and easy to address by wiser choice of > filenames by users. Fortunately, static BPF subprograms don't suffer from this > issues, as they are not independent entities and are neither exposed in BPF > skeleton, nor is lookup-able by any of libbpf APIs (and there is little reason > to do that anyways). > > This and few other things will be the topic of the next set of patches. > I forgot to mention that selftests are relying on Clang fix [0], which is not yet in nightly builds, so kernel-patches CI is currently failing a test. So, when/if testing locally, please make sure to build Clang from latest main. [0] https://reviews.llvm.org/D100362 > Andrii Nakryiko (17): > bpftool: support dumping BTF VAR's "extern" linkage > bpftool: dump more info about DATASEC members > libbpf: suppress compiler warning when using SEC() macro with externs > libbpf: mark BPF subprogs with hidden visibility as static for BPF > verifier > libbpf: allow gaps in BPF program sections to support overriden weak > functions > libbpf: refactor BTF map definition parsing > libbpf: factor out symtab and relos sanity checks > libbpf: make few internal helpers available outside of libbpf.c > libbpf: extend sanity checking ELF symbols with externs validation > libbpf: tighten BTF type ID rewriting with error checking > libbpf: add linker extern resolution support for functions and global > variables > libbpf: support extern resolution for BTF-defined maps in .maps > section > selftests/bpf: use -O0 instead of -Og in selftests builds > selftests/bpf: omit skeleton generation for multi-linked BPF object > files > selftests/bpf: add function linking selftest > selftests/bpf: add global variables linking selftest > sleftests/bpf: add map linking selftest > > tools/bpf/bpftool/btf.c | 30 +- > tools/lib/bpf/bpf_helpers.h | 19 +- > tools/lib/bpf/btf.c | 5 - > tools/lib/bpf/libbpf.c | 370 +++-- > tools/lib/bpf/libbpf_internal.h | 45 + > tools/lib/bpf/linker.c | 1292 ++++++++++++++--- > tools/testing/selftests/bpf/Makefile | 18 +- > .../selftests/bpf/prog_tests/linked_funcs.c | 42 + > .../selftests/bpf/prog_tests/linked_maps.c | 33 + > .../selftests/bpf/prog_tests/linked_vars.c | 43 + > .../selftests/bpf/progs/linked_funcs1.c | 73 + > .../selftests/bpf/progs/linked_funcs2.c | 73 + > .../selftests/bpf/progs/linked_maps1.c | 102 ++ > .../selftests/bpf/progs/linked_maps2.c | 112 ++ > .../selftests/bpf/progs/linked_vars1.c | 60 + > .../selftests/bpf/progs/linked_vars2.c | 61 + > 16 files changed, 2028 insertions(+), 350 deletions(-) > create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_funcs.c > create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_maps.c > create mode 100644 tools/testing/selftests/bpf/prog_tests/linked_vars.c > create mode 100644 tools/testing/selftests/bpf/progs/linked_funcs1.c > create mode 100644 tools/testing/selftests/bpf/progs/linked_funcs2.c > create mode 100644 tools/testing/selftests/bpf/progs/linked_maps1.c > create mode 100644 tools/testing/selftests/bpf/progs/linked_maps2.c > create mode 100644 tools/testing/selftests/bpf/progs/linked_vars1.c > create mode 100644 tools/testing/selftests/bpf/progs/linked_vars2.c > > -- > 2.30.2 >
