2024-03-08 01:08 UTC+0000 ~ Alexei Starovoitov
<alexei.starovoitov@xxxxxxxxx>
> From: Andrii Nakryiko <andrii@xxxxxxxxxx>
>
> LLVM automatically places __arena variables into ".arena.1" ELF section.
> In order to use such global variables bpf program must include definition
> of arena map in ".maps" section, like:
> struct {
> __uint(type, BPF_MAP_TYPE_ARENA);
> __uint(map_flags, BPF_F_MMAPABLE);
> __uint(max_entries, 1000); /* number of pages */
> __ulong(map_extra, 2ull << 44); /* start of mmap() region */
> } arena SEC(".maps");
>
> libbpf recognizes both uses of arena and creates single `struct bpf_map *`
> instance in libbpf APIs.
> ".arena.1" ELF section data is used as initial data image, which is exposed
> through skeleton and bpf_map__initial_value() to the user, if they need to tune
> it before the load phase. During load phase, this initial image is copied over
> into mmap()'ed region corresponding to arena, and discarded.
>
> Few small checks here and there had to be added to make sure this
> approach works with bpf_map__initial_value(), mostly due to hard-coded
> assumption that map->mmaped is set up with mmap() syscall and should be
> munmap()'ed. For arena, .arena.1 can be (much) smaller than maximum
> arena size, so this smaller data size has to be tracked separately.
> Given it is enforced that there is only one arena for entire bpf_object
> instance, we just keep it in a separate field. This can be generalized
> if necessary later.
>
> All global variables from ".arena.1" section are accessible from user space
> via skel->arena->name_of_var.
>
> For bss/data/rodata the skeleton/libbpf perform the following sequence:
> 1. addr = mmap(MAP_ANONYMOUS)
> 2. user space optionally modifies global vars
> 3. map_fd = bpf_create_map()
> 4. bpf_update_map_elem(map_fd, addr) // to store values into the kernel
> 5. mmap(addr, MAP_FIXED, map_fd)
> after step 5 user spaces see the values it wrote at step 2 at the same addresses
>
> arena doesn't support update_map_elem. Hence skeleton/libbpf do:
> 1. addr = malloc(sizeof SEC ".arena.1")
> 2. user space optionally modifies global vars
> 3. map_fd = bpf_create_map(MAP_TYPE_ARENA)
> 4. real_addr = mmap(map->map_extra, MAP_SHARED | MAP_FIXED, map_fd)
> 5. memcpy(real_addr, addr) // this will fault-in and allocate pages
>
> At the end look and feel of global data vs __arena global data is the same from
> bpf prog pov.
>
> Another complication is:
> struct {
> __uint(type, BPF_MAP_TYPE_ARENA);
> } arena SEC(".maps");
>
> int __arena foo;
> int bar;
>
> ptr1 = &foo; // relocation against ".arena.1" section
> ptr2 = &arena; // relocation against ".maps" section
> ptr3 = &bar; // relocation against ".bss" section
>
> Fo the kernel ptr1 and ptr2 has point to the same arena's map_fd
> while ptr3 points to a different global array's map_fd.
> For the verifier:
> ptr1->type == unknown_scalar
> ptr2->type == const_ptr_to_map
> ptr3->type == ptr_to_map_value
>
> After verification, from JIT pov all 3 ptr-s are normal ld_imm64 insns.
>
> Signed-off-by: Andrii Nakryiko <andrii@xxxxxxxxxx>
> Signed-off-by: Alexei Starovoitov <ast@xxxxxxxxxx>
> ---
> tools/bpf/bpftool/gen.c | 13 +++++
> tools/lib/bpf/libbpf.c | 118 ++++++++++++++++++++++++++++++++++++----
> tools/lib/bpf/libbpf.h | 2 +-
> 3 files changed, 120 insertions(+), 13 deletions(-)
>
> diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c
> index a3d72be347b0..4fa4ade1ce74 100644
> --- a/tools/bpf/bpftool/gen.c
> +++ b/tools/bpf/bpftool/gen.c
> @@ -120,6 +120,12 @@ static bool get_datasec_ident(const char *sec_name, char *buf, size_t buf_sz)
> static const char *pfxs[] = { ".data", ".rodata", ".bss", ".kconfig" };
> int i, n;
>
> + /* recognize hard coded LLVM section name */
> + if (strcmp(sec_name, ".arena.1") == 0) {
> + /* this is the name to use in skeleton */
> + snprintf(buf, buf_sz, "arena");
> + return true;
> + }
> for (i = 0, n = ARRAY_SIZE(pfxs); i < n; i++) {
> const char *pfx = pfxs[i];
>
> @@ -250,6 +256,13 @@ static const struct btf_type *find_type_for_map(struct btf *btf, const char *map
>
> static bool is_mmapable_map(const struct bpf_map *map, char *buf, size_t sz)
> {
> + size_t tmp_sz;
> +
> + if (bpf_map__type(map) == BPF_MAP_TYPE_ARENA && bpf_map__initial_value(map, &tmp_sz)) {
> + snprintf(buf, sz, "arena");
> + return true;
> + }
> +
> if (!bpf_map__is_internal(map) || !(bpf_map__map_flags(map) & BPF_F_MMAPABLE))
> return false;
>
For the bpftool changes:
Acked-by: Quentin Monnet <quentin@xxxxxxxxxxxxx>
