On Fri, May 6, 2022 at 3:35 PM Andrii Nakryiko <andrii.nakryiko@xxxxxxxxx> wrote: > > On Thu, Apr 28, 2022 at 2:11 PM Joanne Koong <joannelkoong@xxxxxxxxx> wrote: > > > > This patchset implements the basics of dynamic pointers in bpf. > > > > A dynamic pointer (struct bpf_dynptr) is a pointer that stores extra metadata > > alongside the address it points to. This abstraction is useful in bpf, given > > that every memory access in a bpf program must be safe. The verifier and bpf > > helper functions can use the metadata to enforce safety guarantees for things > > such as dynamically sized strings and kernel heap allocations. > > > > From the program side, the bpf_dynptr is an opaque struct and the verifier > > will enforce that its contents are never written to by the program. > > It can only be written to through specific bpf helper functions. > > > > There are several uses cases for dynamic pointers in bpf programs. A list of > > some are: dynamically sized ringbuf reservations without any extra memcpys, > > dynamic string parsing and memory comparisons, dynamic memory allocations that > > can be persisted in a map, and dynamic parsing of sk_buff and xdp_md packet > > data. > > > > At a high-level, the patches are as follows: > > 1/6 - Adds MEM_UNINIT as a bpf_type_flag > > 2/6 - Adds verifier support for dynptrs and implements malloc dynptrs > > 3/6 - Adds dynptr support for ring buffers > > 4/6 - Adds bpf_dynptr_read and bpf_dynptr_write > > 5/6 - Adds dynptr data slices (ptr to underlying dynptr memory) > > 6/6 - Tests to check that verifier rejects certain fail cases and passes > > certain success cases > > > > This is the first dynptr patchset in a larger series. The next series of > > patches will add persisting dynamic memory allocations in maps, parsing packet > > data through dynptrs, convenience helpers for using dynptrs as iterators, and > > more helper functions for interacting with strings and memory dynamically. > > > > Changelog: > > ---------- > > v3 -> v2: > > v2: https://lore.kernel.org/bpf/20220416063429.3314021-1-joannelkoong@xxxxxxxxx/ > > > > * Reorder patches (move ringbuffer patch to be right after the verifier + malloc > > dynptr patchset) > > * Remove local type dynptrs (Andrii + Alexei) > > * Mark stack slot as STACK_MISC after any writes into a dynptr instead of > > * explicitly prohibiting writes (Alexei) > > * Pass number of slots, not memory size to is_spi_bounds_valid (Kumar) > > * Check reference leaks by adding dynptr id to state->refs instead of checking > > stack slots (Alexei) > > > > There seems to be test_progs-no_alu32 failure in CI ([0]), please take a look > > [0] https://github.com/kernel-patches/bpf/runs/6223168213?check_suite_focus=true#step:6:6430 > I'll look into this and fix it for v4. thanks for taking the time to review all these dynptr changes in all the versions, Andrii! > > v1 -> v2: > > v1: https://lore.kernel.org/bpf/20220402015826.3941317-1-joannekoong@xxxxxx/ > > > > 1/7 - > > * Remove ARG_PTR_TO_MAP_VALUE_UNINIT alias and use > > ARG_PTR_TO_MAP_VALUE | MEM_UNINIT directly (Andrii) > > * Drop arg_type_is_mem_ptr() wrapper function (Andrii) > > 2/7 - > > * Change name from MEM_RELEASE to OBJ_RELEASE (Andrii) > > * Use meta.release_ref instead of ref_obj_id != 0 to determine whether > > to release reference (Kumar) > > * Drop type_is_release_mem() wrapper function (Andrii) > > 3/7 - > > * Add checks for nested dynptrs edge-cases, which could lead to corrupt > > * writes of the dynptr stack variable. > > * Add u64 flags to bpf_dynptr_from_mem() and bpf_dynptr_alloc() (Andrii) > > * Rename from bpf_malloc/bpf_free to bpf_dynptr_alloc/bpf_dynptr_put > > (Alexei) > > * Support alloc flag __GFP_ZERO (Andrii) > > * Reserve upper 8 bits in dynptr size and offset fields instead of > > reserving just the upper 4 bits (Andrii) > > * Allow dynptr zero-slices (Andrii) > > * Use the highest bit for is_rdonly instead of the 28th bit (Andrii) > > * Rename check_* functions to is_* functions for better readability > > (Andrii) > > * Add comment for code that checks the spi bounds (Andrii) > > 4/7 - > > * Fix doc description for bpf_dynpt_read (Toke) > > * Move bpf_dynptr_check_off_len() from function patch 1 to here (Andrii) > > 5/7 - > > * When finding the id for the dynptr to associate the data slice with, > > look for dynptr arg instead of assuming it is BPF_REG_1. > > 6/7 - > > * Add __force when casting from unsigned long to void * (kernel test robot) > > * Expand on docs for ringbuf dynptr APIs (Andrii) > > 7/7 - > > * Use table approach for defining test programs and error messages (Andrii) > > * Print out full log if there’s an error (Andrii) > > * Use bpf_object__find_program_by_name() instead of specifying > > program name as a string (Andrii) > > * Add 6 extra cases: invalid_nested_dynptrs1, invalid_nested_dynptrs2, > > invalid_ref_mem1, invalid_ref_mem2, zero_slice_access, > > and test_alloc_zero_bytes > > * Add checking for edge cases (eg allocing with invalid flags) > > > > > > Joanne Koong (6): > > bpf: Add MEM_UNINIT as a bpf_type_flag > > bpf: Add verifier support for dynptrs and implement malloc dynptrs > > bpf: Dynptr support for ring buffers > > bpf: Add bpf_dynptr_read and bpf_dynptr_write > > bpf: Add dynptr data slices > > bpf: Dynptr tests > > > > include/linux/bpf.h | 103 +++- > > include/linux/bpf_verifier.h | 21 + > > include/uapi/linux/bpf.h | 96 +++ > > kernel/bpf/helpers.c | 169 +++++- > > kernel/bpf/ringbuf.c | 71 +++ > > kernel/bpf/verifier.c | 329 +++++++++- > > scripts/bpf_doc.py | 2 + > > tools/include/uapi/linux/bpf.h | 96 +++ > > .../testing/selftests/bpf/prog_tests/dynptr.c | 132 ++++ > > .../testing/selftests/bpf/progs/dynptr_fail.c | 574 ++++++++++++++++++ > > .../selftests/bpf/progs/dynptr_success.c | 218 +++++++ > > 11 files changed, 1774 insertions(+), 37 deletions(-) > > create mode 100644 tools/testing/selftests/bpf/prog_tests/dynptr.c > > create mode 100644 tools/testing/selftests/bpf/progs/dynptr_fail.c > > create mode 100644 tools/testing/selftests/bpf/progs/dynptr_success.c > > > > -- > > 2.30.2 > >
