Add support to BPF verifier to track and support register spill/fill to/from
stack regardless if it was done through read-only R10 register (which is the
only form supported today), or through a general register after copying R10
into it, while also potentially modifying offset.
Once we add register this generic spill/fill support to precision
backtracking, we can take advantage of it to stop doing eager STACK_ZERO
conversion on register spill. Instead we can rely on (im)precision of spilled
const zero register to improve verifier state pruning efficiency. This
situation of using const zero register to initialize stack slots is very
common with __builtin_memset() usage or just zero-initializing variables on
the stack, and it causes unnecessary state duplication, as that STACK_ZERO
knowledge is often not necessary for correctness, as those zero values are
never used in precise context. Thus, relying on register imprecision helps
tremendously, especially in real-world BPF programs.
To make spilled const zero register behave completely equivalently to
STACK_ZERO, we need to improve few other small pieces, which is done in the
second part of the patch set. See individual patches for details. There are
also two small bug fixes spotted during STACK_ZERO debugging.
Andrii Nakryiko (7):
bpf: use common jump (instruction) history across all states
bpf: support non-r10 register spill/fill to/from stack in precision
tracking
bpf: enforce precision for r0 on callback return
bpf: fix check for attempt to corrupt spilled pointer
bpf: preserve STACK_ZERO slots on partial reg spills
bpf: preserve constant zero when doing partial register restore
bpf: track aligned STACK_ZERO cases as imprecise spilled registers
include/linux/bpf_verifier.h | 34 ++-
kernel/bpf/verifier.c | 274 ++++++++++--------
.../bpf/progs/verifier_subprog_precision.c | 83 +++++-
.../testing/selftests/bpf/verifier/precise.c | 38 ++-
4 files changed, 285 insertions(+), 144 deletions(-)
--
2.34.1
