On Tue, Mar 15, 2022 at 3:15 PM Delyan Kratunov <delyank@xxxxxx> wrote:
>
> In the quest for ever more modularity, a new need has arisen - the ability to
> access data associated with a BPF library from a corresponding userspace library.
> The catch is that we don't want the userspace library to know about the structure of the
> final BPF object that the BPF library is linked into.
>
> In pursuit of this modularity, this patch series introduces *subskeletons.*
> Subskeletons are similar in use and design to skeletons with a couple of differences:
>
> 1. The generated storage types do not rely on contiguous storage for the library's
> variables because they may be interspersed randomly throughout the final BPF object's sections.
>
> 2. Subskeletons do not own objects and instead require a loaded bpf_object* to
> be passed at runtime in order to be initialized. By extension, symbols are resolved at
> runtime by parsing the final object's BTF. This has the interesting effect that the same
> userspace code can interoperate with the library BPF code *linked into different final objects.*
>
> 3. Subskeletons allow access to all global variables, programs, and custom maps. They also expose
> the internal maps *of the final object*. This allows bpf_var_skeleton objects to contain a bpf_map**
> instead of a section name.
>
> Changes since v2:
> - Reuse SEC_NAME strict mode flag
> - Init bpf_map->btf_value_type_id on open for internal maps *and* user BTF maps
> - Test custom section names (.data.foo) and overlapping kconfig externs between the final object and the library
> - Minor review comments in gen.c & libbpf.c
>
> Changes since v1:
> - Introduced new strict mode knob for single-routine-in-.text compatibility behavior, which
> disproportionately affects library objects. bpftool works in 1.0 mode so subskeleton generation
> doesn't have to worry about this now.
> - Made bpf_map_btf_value_type_id available earlier and used it wherever applicable.
> - Refactoring in bpftool gen.c per review comments.
> - Subskels now use typeof() for array and func proto globals to avoid the need for runtime split btf.
> - Expanded the subskeleton test to include arrays, custom maps, extern maps, weak symbols, and kconfigs.
> - selftests/bpf/Makefile now generates a subskel.h for every skel.h it would make.
>
> For reference, here is a shortened subskeleton header:
>
> #ifndef __TEST_SUBSKELETON_LIB_SUBSKEL_H__
> #define __TEST_SUBSKELETON_LIB_SUBSKEL_H__
>
> struct test_subskeleton_lib {
> struct bpf_object *obj;
> struct bpf_object_subskeleton *subskel;
> struct {
> struct bpf_map *map2;
> struct bpf_map *map1;
> struct bpf_map *data;
> struct bpf_map *rodata;
> struct bpf_map *bss;
> struct bpf_map *kconfig;
> } maps;
> struct {
> struct bpf_program *lib_perf_handler;
> } progs;
> struct test_subskeleton_lib__data {
> int *var6;
> int *var2;
> int *var5;
> } data;
> struct test_subskeleton_lib__rodata {
> int *var1;
> } rodata;
> struct test_subskeleton_lib__bss {
> struct {
> int var3_1;
> __s64 var3_2;
> } *var3;
> int *libout1;
> typeof(int[4]) *var4;
> typeof(int (*)()) *fn_ptr;
> } bss;
> struct test_subskeleton_lib__kconfig {
> _Bool *CONFIG_BPF_SYSCALL;
> } kconfig;
>
> static inline struct test_subskeleton_lib *
> test_subskeleton_lib__open(const struct bpf_object *src)
> {
> struct test_subskeleton_lib *obj;
> struct bpf_object_subskeleton *s;
> int err;
>
> ...
> s = (struct bpf_object_subskeleton *)calloc(1, sizeof(*s));
> ...
>
> s->var_cnt = 9;
> ...
>
> s->vars[0].name = "var6";
> s->vars[0].map = &obj->maps.data;
> s->vars[0].addr = (void**) &obj->data.var6;
> ...
>
> /* maps */
> ...
>
> /* programs */
> s->prog_cnt = 1;
> ...
>
> err = bpf_object__open_subskeleton(s);
> ...
> return obj;
> }
> #endif /* __TEST_SUBSKELETON_LIB_SUBSKEL_H__ */
>
> Delyan Kratunov (5):
> libbpf: .text routines are subprograms in strict mode
> libbpf: init btf_{key,value}_type_id on internal map open
> libbpf: add subskeleton scaffolding
> bpftool: add support for subskeletons
> selftests/bpf: test subskeleton functionality
>
CI fails at test #18 (btf) ([0]), please take a look. I was able to
repro it locally as well.
[0] https://github.com/kernel-patches/bpf/runs/5562170931?check_suite_focus=true
Given you are going to resubmit, I left a few more nits that I'd
otherwise fix while applying.
> .../bpf/bpftool/Documentation/bpftool-gen.rst | 25 +
> tools/bpf/bpftool/bash-completion/bpftool | 14 +-
> tools/bpf/bpftool/gen.c | 595 +++++++++++++++---
> tools/lib/bpf/libbpf.c | 155 ++++-
> tools/lib/bpf/libbpf.h | 29 +
> tools/lib/bpf/libbpf.map | 2 +
> tools/lib/bpf/libbpf_legacy.h | 4 +-
> tools/testing/selftests/bpf/.gitignore | 1 +
> tools/testing/selftests/bpf/Makefile | 12 +-
> .../selftests/bpf/prog_tests/subskeleton.c | 78 +++
> .../selftests/bpf/progs/test_subskeleton.c | 28 +
> .../bpf/progs/test_subskeleton_lib.c | 61 ++
> .../bpf/progs/test_subskeleton_lib2.c | 16 +
> 13 files changed, 910 insertions(+), 110 deletions(-)
> create mode 100644 tools/testing/selftests/bpf/prog_tests/subskeleton.c
> create mode 100644 tools/testing/selftests/bpf/progs/test_subskeleton.c
> create mode 100644 tools/testing/selftests/bpf/progs/test_subskeleton_lib.c
> create mode 100644 tools/testing/selftests/bpf/progs/test_subskeleton_lib2.c
>
> --
> 2.34.1
