From: Alexei Starovoitov <ast@xxxxxxxxxx>
v3->v4:
- patch 3 got much cleaner after BPF_KPTR_RCU was removed as suggested by David.
- make KF_RCU stronger and require that bpf program checks for NULL
before passing such pointers into kfunc. The prog has to do that anyway
to access fields and it aligns with BTF_TYPE_SAFE_RCU allowlist.
- New patch 6: refactor RCU enforcement in the verifier.
The patches 2,3,6 are part of one feature.
The 2 and 3 alone are incomplete, since RCU pointers are barely useful
without bpf_rcu_read_lock/unlock in GCC compiled kernel.
Even if GCC lands support for btf_type_tag today it will take time
to mandate that version for kernel builds. Hence go with allow list
approach. See patch 6 for details.
This allows to start strict enforcement of TRUSTED | UNTRUSTED
in one part of PTR_TO_BTF_ID accesses.
One step closer to KF_TRUSTED_ARGS by default.
v2->v3:
- Instead of requiring bpf progs to tag fields with __kptr_rcu
teach the verifier to infer RCU properties based on the type.
BPF_KPTR_RCU becomes kernel internal type of struct btf_field.
- Add patch 2 to tag cgroups and dfl_cgrp as trusted.
That bug was spotted by BPF CI on clang compiler kernels,
since patch 3 is doing:
static bool in_rcu_cs(struct bpf_verifier_env *env)
{
return env->cur_state->active_rcu_lock || !env->prog->aux->sleepable;
}
which makes all non-sleepable programs behave like they have implicit
rcu_read_lock around them. Which is the case in practice.
It was fine on gcc compiled kernels where task->cgroup deference was producing
PTR_TO_BTF_ID, but on clang compiled kernels task->cgroup deference was
producing PTR_TO_BTF_ID | MEM_RCU | MAYBE_NULL, which is more correct,
but selftests were failing. Patch 2 fixes this discrepancy.
With few more patches like patch 2 we can make KF_TRUSTED_ARGS default
for kfuncs and helpers.
- Add comment in selftest patch 5 that it's verifier only check.
v1->v2:
Instead of agressively allow dereferenced kptr_rcu pointers into KF_TRUSTED_ARGS
kfuncs only allow them into KF_RCU funcs.
The KF_RCU flag is a weaker version of KF_TRUSTED_ARGS. The kfuncs marked with
KF_RCU expect either PTR_TRUSTED or MEM_RCU arguments. The verifier guarantees
that the objects are valid and there is no use-after-free, but the pointers
maybe NULL and pointee object's reference count could have reached zero, hence
kfuncs must do != NULL check and consider refcnt==0 case when accessing such
arguments.
No changes in patch 1.
Patches 2,3,4 adjusted with above behavior.
v1:
The __kptr_ref turned out to be too limited, since any "trusted" pointer access
requires bpf_kptr_xchg() which is impractical when the same pointer needs
to be dereferenced by multiple cpus.
The __kptr "untrusted" only access isn't very useful in practice.
Rename __kptr to __kptr_untrusted with eventual goal to deprecate it,
and rename __kptr_ref to __kptr, since that looks to be more common use of kptrs.
Introduce __kptr_rcu that can be directly dereferenced and used similar
to native kernel C code.
Once bpf_cpumask and task_struct kfuncs are converted to observe RCU GP
when refcnt goes to zero, both __kptr and __kptr_untrusted can be deprecated
and __kptr_rcu can become the only __kptr tag.
Alexei Starovoitov (6):
bpf: Rename __kptr_ref -> __kptr and __kptr -> __kptr_untrusted.
bpf: Mark cgroups and dfl_cgrp fields as trusted.
bpf: Introduce kptr_rcu.
selftests/bpf: Add a test case for kptr_rcu.
selftests/bpf: Tweak cgroup kfunc test.
bpf: Refactor RCU enforcement in the verifier.
Documentation/bpf/bpf_design_QA.rst | 4 +-
Documentation/bpf/cpumasks.rst | 4 +-
Documentation/bpf/kfuncs.rst | 14 +-
include/linux/bpf.h | 2 +-
include/linux/bpf_verifier.h | 1 -
include/linux/btf.h | 2 +-
kernel/bpf/btf.c | 19 +-
kernel/bpf/cpumask.c | 40 ++--
kernel/bpf/helpers.c | 6 +-
kernel/bpf/verifier.c | 213 +++++++++++++-----
net/bpf/test_run.c | 3 +-
tools/lib/bpf/bpf_helpers.h | 2 +-
.../bpf/prog_tests/cgrp_local_storage.c | 14 +-
.../selftests/bpf/prog_tests/rcu_read_lock.c | 16 +-
tools/testing/selftests/bpf/progs/cb_refs.c | 2 +-
.../selftests/bpf/progs/cgrp_kfunc_common.h | 2 +-
.../selftests/bpf/progs/cgrp_kfunc_failure.c | 2 +-
.../selftests/bpf/progs/cgrp_kfunc_success.c | 12 +-
.../selftests/bpf/progs/cgrp_ls_sleepable.c | 4 +-
.../selftests/bpf/progs/cpumask_common.h | 2 +-
.../selftests/bpf/progs/cpumask_failure.c | 2 +-
.../selftests/bpf/progs/jit_probe_mem.c | 2 +-
tools/testing/selftests/bpf/progs/lru_bug.c | 2 +-
tools/testing/selftests/bpf/progs/map_kptr.c | 16 +-
.../selftests/bpf/progs/map_kptr_fail.c | 10 +-
.../bpf/progs/nested_trust_failure.c | 2 +-
.../selftests/bpf/progs/rcu_read_lock.c | 6 +-
.../selftests/bpf/progs/task_kfunc_common.h | 2 +-
tools/testing/selftests/bpf/test_verifier.c | 22 +-
tools/testing/selftests/bpf/verifier/calls.c | 4 +-
.../testing/selftests/bpf/verifier/map_kptr.c | 2 +-
31 files changed, 283 insertions(+), 151 deletions(-)
--
2.39.2
