This series implements support for sleepable uprobe programs. Key work is in patches 2 and 3, the rest is plumbing and tests. The main observation is that the only obstacle in the way of sleepable uprobe programs is not the uprobe infrastructure, which already runs in a user context, but the rcu usage around bpf_prog_array. Details are in patch 2 but the tl;dr is that we chain trace_tasks and normal rcu grace periods when releasing to array to accommodate users of either rcu type. This introduces latency for non-sleepable users (kprobe, tp) but that's deemed acceptable, given recent benchmarks by Andrii [1]. We're a couple of orders of magnitude under the rate of bpf_prog_array churn that would raise flags (~1MM/s per Paul). [1]: https://lore.kernel.org/bpf/CAEf4BzbpjN6ca7D9KOTiFPOoBYkciYvTz0UJNp5c-_3ptm=Mrg@xxxxxxxxxxxxxx/ v1 -> v2: * Fix lockdep annotations in bpf_prog_run_array_sleepable * Chain rcu grace periods only for perf_event-attached programs. This limits the additional latency on the free path to use cases where we know it won't be a problem. * Add tests calling helpers only available in sleepable programs. * Remove kprobe.s support from libbpf. Delyan Kratunov (5): bpf: move bpf_prog to bpf.h bpf: implement sleepable uprobes by chaining tasks_trace and normal rcu bpf: allow sleepable uprobe programs to attach libbpf: add support for sleepable kprobe and uprobe programs selftests/bpf: add tests for sleepable kprobes and uprobes include/linux/bpf.h | 93 +++++++++++++++++++ include/linux/filter.h | 34 ------- include/linux/trace_events.h | 1 + kernel/bpf/core.c | 15 +++ kernel/bpf/verifier.c | 4 +- kernel/events/core.c | 16 ++-- kernel/trace/bpf_trace.c | 27 +++++- kernel/trace/trace_uprobe.c | 4 +- tools/lib/bpf/libbpf.c | 6 +- .../selftests/bpf/prog_tests/attach_probe.c | 51 +++++++++- .../selftests/bpf/progs/test_attach_probe.c | 58 ++++++++++++ 11 files changed, 260 insertions(+), 49 deletions(-) -- 2.35.1
