This set implements the second part of the exceptions set, with support for releasing resources at runtime during the unwinding phase. This allows programs to throw an exception at any point of time and terminate their execution immediately. Currently, any acquired resources held by the program will cause a thrown exception to fail during verification. This is because safely unwinding the stack requires releasing these resources which represent kernel objects and locks. Not doing this and continuing the stack unwinding will destroy the stack frames containing such objects and will lead to all kinds of issues and the violation of BPF's safety properties. Note that while the current mechanism only supports throwing exceptions synchronously, the unwinding mechanism only requires a valid frame descriptor to perform the cleanup. Thus, in a followup, we can build on top of this series to introduce support for preempting execution of BPF programs and terminating them, in cases where termination is not statically provable and the program exceeds some runtime threshold. This can also allow holding multiple locks at the same time, detecting deadlocks and aborting programs, etc. In this set, we implement support that allows the kernel to release such resources when performing the unwinding step for the program and aborting it. To enable this, the kernel needs to be made aware about the layout of the program stack (for each BPF frame) and the type of each kernel object residing therein. This information is retrieved at runtime by tying this metadata to the program counter. Everytime a bpf_throw call is processed by the verifier, it generates a frame descriptor for the caller and all of its caller frames, and ties them to the program counter. At runtime, the kernel will only process unwinding requests for such program counters, and this mapping of frame descriptors to the PC will allow their discovery of resources for each frame. Note that at the same program point, there cannot be a case where the verifier may have to produce distinct frame descriptors. If such a case is encountered, then the verification will fail. This is an uncommon case where depending on the program path taken at runtime, the same stack slot may contain pointers of different types. In most of these examples, the program would not pass the verifier anyway, since the value that has to be freed would be lost in verifier state and cannot be recovered. A special provision is made for cases where the stack slot contains NULL or a pointer in two different program paths, it is quite common for such a case to occur where a resource may be acquired conditionally, and the release occurs later in the program guarded by the same conditional. Notes ===== * Releasing bpf_spin_lock, RCU read locks is not supported in the RFC, but will be done as a follow up or added to the next revision on top of this set. * A few known rough edges/minor bugs which will be fixed in the next version. * Adding more tests for corner cases. Kumar Kartikeya Dwivedi (14): bpf: Mark subprogs as throw reachable before do_check pass bpf: Process global subprog's exception propagation selftests/bpf: Add test for throwing global subprog with acquired refs bpf: Refactor check_pseudo_btf_id's BTF reference bump bpf: Implement BPF exception frame descriptor generation bpf: Adjust frame descriptor pc on instruction patching bpf: Use hidden subprog trampoline for bpf_throw bpf: Compute used callee saved registers for subprogs bpf, x86: Fix up pc offsets for frame descriptor entries bpf, x86: Implement runtime resource cleanup for exceptions bpf: Release references in verifier state when throwing exceptions bpf: Register cleanup dtors for runtime unwinding bpf: Make bpf_throw available to all program types selftests/bpf: Add tests for exceptions runtime cleanup arch/x86/net/bpf_jit_comp.c | 116 ++- drivers/hid/bpf/hid_bpf_dispatch.c | 17 + include/linux/bpf.h | 57 ++ include/linux/bpf_verifier.h | 9 +- include/linux/btf.h | 10 +- include/linux/filter.h | 3 + kernel/bpf/btf.c | 11 +- kernel/bpf/core.c | 18 + kernel/bpf/cpumask.c | 3 +- kernel/bpf/helpers.c | 165 +++- kernel/bpf/verifier.c | 714 +++++++++++++++++- kernel/trace/bpf_trace.c | 16 + net/bpf/test_run.c | 4 +- net/core/filter.c | 5 + net/netfilter/nf_conntrack_bpf.c | 14 +- net/xfrm/xfrm_state_bpf.c | 16 + tools/testing/selftests/bpf/DENYLIST.aarch64 | 1 + tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../bpf/prog_tests/exceptions_cleanup.c | 65 ++ .../selftests/bpf/progs/exceptions_cleanup.c | 468 ++++++++++++ .../bpf/progs/exceptions_cleanup_fail.c | 154 ++++ .../selftests/bpf/progs/exceptions_fail.c | 38 +- 22 files changed, 1817 insertions(+), 88 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/exceptions_cleanup.c create mode 100644 tools/testing/selftests/bpf/progs/exceptions_cleanup.c create mode 100644 tools/testing/selftests/bpf/progs/exceptions_cleanup_fail.c base-commit: 77326a4a06e1e97432322f403cb439880871d34d -- 2.40.1
