From: Alexei Starovoitov <ast@xxxxxxxxxx>
Since bpf hash map was converted to use bpf_mem_alloc it is safe to use
from tracing programs and in RT kernels.
But per-cpu hash map is still using dynamic allocation for per-cpu map
values, hence keep the warning for this map type.
In the future alloc_percpu_gfp can be front-end-ed with bpf_mem_cache
and this restriction will be completely lifted.
perf_event (NMI) bpf programs have to use preallocated hash maps,
because free_htab_elem() is using call_rcu which might crash if re-entered.
Sleepable bpf programs have to use preallocated hash maps, because
life time of the map elements is not protected by rcu_read_lock/unlock.
This restriction can be lifted in the future as well.
Signed-off-by: Alexei Starovoitov <ast@xxxxxxxxxx>
---
kernel/bpf/verifier.c | 31 ++++++++++++++++++++++---------
1 file changed, 22 insertions(+), 9 deletions(-)
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index a20d7736a5b2..90d70304c6f1 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -12566,10 +12566,12 @@ static int check_map_prog_compatibility(struct bpf_verifier_env *env,
* For programs attached to PERF events this is mandatory as the
* perf NMI can hit any arbitrary code sequence.
*
- * All other trace types using preallocated hash maps are unsafe as
- * well because tracepoint or kprobes can be inside locked regions
- * of the memory allocator or at a place where a recursion into the
- * memory allocator would see inconsistent state.
+ * All other trace types using non-preallocated per-cpu hash maps are
+ * unsafe as well because tracepoint or kprobes can be inside locked
+ * regions of the per-cpu memory allocator or at a place where a
+ * recursion into the per-cpu memory allocator would see inconsistent
+ * state. Non per-cpu hash maps are using bpf_mem_alloc-tor which is
+ * safe to use from kprobe/fentry and in RT.
*
* On RT enabled kernels run-time allocation of all trace type
* programs is strictly prohibited due to lock type constraints. On
@@ -12579,15 +12581,26 @@ static int check_map_prog_compatibility(struct bpf_verifier_env *env,
*/
if (is_tracing_prog_type(prog_type) && !is_preallocated_map(map)) {
if (prog_type == BPF_PROG_TYPE_PERF_EVENT) {
+ /* perf_event bpf progs have to use preallocated hash maps
+ * because non-prealloc is still relying on call_rcu to free
+ * elements.
+ */
verbose(env, "perf_event programs can only use preallocated hash map\n");
return -EINVAL;
}
- if (IS_ENABLED(CONFIG_PREEMPT_RT)) {
- verbose(env, "trace type programs can only use preallocated hash map\n");
- return -EINVAL;
+ if (map->map_type == BPF_MAP_TYPE_PERCPU_HASH ||
+ (map->inner_map_meta &&
+ map->inner_map_meta->map_type == BPF_MAP_TYPE_PERCPU_HASH)) {
+ if (IS_ENABLED(CONFIG_PREEMPT_RT)) {
+ verbose(env,
+ "trace type programs can only use preallocated per-cpu hash map\n");
+ return -EINVAL;
+ }
+ WARN_ONCE(1, "trace type BPF program uses run-time allocation\n");
+ verbose(env,
+ "trace type programs with run-time allocated per-cpu hash maps are unsafe."
+ " Switch to preallocated hash maps.\n");
}
- WARN_ONCE(1, "trace type BPF program uses run-time allocation\n");
- verbose(env, "trace type programs with run-time allocated hash maps are unsafe. Switch to preallocated hash maps.\n");
}
if (map_value_has_spin_lock(map)) {
--
2.30.2
