From: Alexei Starovoitov <ast@xxxxxxxxxx>
v2->v3:
- Instead of requiring bpf progs to tag fields with __kptr_rcu
teach the verifier to infer RCU properties based on the type.
BPF_KPTR_RCU becomes kernel internal type of struct btf_field.
- Add patch 2 to tag cgroups and dfl_cgrp as trusted.
That bug was spotted by BPF CI on clang compiler kernels,
since patch 3 is doing:
static bool in_rcu_cs(struct bpf_verifier_env *env)
{
return env->cur_state->active_rcu_lock || !env->prog->aux->sleepable;
}
which makes all non-sleepable programs behave like they have implicit
rcu_read_lock around them. Which is the case in practice.
It was fine on gcc compiled kernels where task->cgroup deference was producing
PTR_TO_BTF_ID, but on clang compiled kernels task->cgroup deference was
producing PTR_TO_BTF_ID | MEM_RCU | MAYBE_NULL, which is more correct,
but selftests were failing. Patch 2 fixes this discrepancy.
With few more patches like patch 2 we can make KF_TRUSTED_ARGS default
for kfuncs and helpers.
- Add comment in selftest patch 5 that it's verifier only check.
v1->v2:
Instead of agressively allow dereferenced kptr_rcu pointers into KF_TRUSTED_ARGS
kfuncs only allow them into KF_RCU funcs.
The KF_RCU flag is a weaker version of KF_TRUSTED_ARGS. The kfuncs marked with
KF_RCU expect either PTR_TRUSTED or MEM_RCU arguments. The verifier guarantees
that the objects are valid and there is no use-after-free, but the pointers
maybe NULL and pointee object's reference count could have reached zero, hence
kfuncs must do != NULL check and consider refcnt==0 case when accessing such
arguments.
No changes in patch 1.
Patches 2,3,4 adjusted with above behavior.
v1:
The __kptr_ref turned out to be too limited, since any "trusted" pointer access
requires bpf_kptr_xchg() which is impractical when the same pointer needs
to be dereferenced by multiple cpus.
The __kptr "untrusted" only access isn't very useful in practice.
Rename __kptr to __kptr_untrusted with eventual goal to deprecate it,
and rename __kptr_ref to __kptr, since that looks to be more common use of kptrs.
Introduce __kptr_rcu that can be directly dereferenced and used similar
to native kernel C code.
Once bpf_cpumask and task_struct kfuncs are converted to observe RCU GP
when refcnt goes to zero, both __kptr and __kptr_untrusted can be deprecated
and __kptr_rcu can become the only __kptr tag.
Alexei Starovoitov (5):
bpf: Rename __kptr_ref -> __kptr and __kptr -> __kptr_untrusted.
bpf: Mark cgroups and dfl_cgrp fields as trusted.
bpf: Introduce kptr_rcu.
selftests/bpf: Add a test case for kptr_rcu.
selftests/bpf: Tweak cgroup kfunc test.
Documentation/bpf/bpf_design_QA.rst | 4 +-
Documentation/bpf/cpumasks.rst | 4 +-
Documentation/bpf/kfuncs.rst | 13 ++++---
include/linux/bpf.h | 15 +++++---
include/linux/btf.h | 2 +-
kernel/bpf/btf.c | 20 +++++++++-
kernel/bpf/helpers.c | 7 ++--
kernel/bpf/syscall.c | 4 ++
kernel/bpf/verifier.c | 38 +++++++++++++------
net/bpf/test_run.c | 3 +-
tools/lib/bpf/bpf_helpers.h | 2 +-
tools/testing/selftests/bpf/progs/cb_refs.c | 2 +-
.../selftests/bpf/progs/cgrp_kfunc_common.h | 2 +-
.../selftests/bpf/progs/cgrp_kfunc_failure.c | 2 +-
.../selftests/bpf/progs/cgrp_kfunc_success.c | 7 +++-
.../selftests/bpf/progs/cpumask_common.h | 2 +-
.../selftests/bpf/progs/jit_probe_mem.c | 2 +-
tools/testing/selftests/bpf/progs/lru_bug.c | 2 +-
tools/testing/selftests/bpf/progs/map_kptr.c | 18 ++++++++-
.../selftests/bpf/progs/map_kptr_fail.c | 10 ++---
.../selftests/bpf/progs/task_kfunc_common.h | 2 +-
tools/testing/selftests/bpf/test_verifier.c | 22 +++++------
tools/testing/selftests/bpf/verifier/calls.c | 2 +-
.../testing/selftests/bpf/verifier/map_kptr.c | 2 +-
24 files changed, 125 insertions(+), 62 deletions(-)
--
2.30.2
