On Thu, Oct 10, 2019 at 05:19:01PM +0000, Alexei Starovoitov wrote:
> On 10/10/19 12:36 AM, Peter Zijlstra wrote:
> > On Wed, Oct 09, 2019 at 11:19:16PM -0700, Song Liu wrote:
> >> bpf stackmap with build-id lookup (BPF_F_STACK_BUILD_ID) can trigger A-A
> >> deadlock on rq_lock():
> >>
> >> rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
> >> [...]
> >> Call Trace:
> >> try_to_wake_up+0x1ad/0x590
> >> wake_up_q+0x54/0x80
> >> rwsem_wake+0x8a/0xb0
> >> bpf_get_stack+0x13c/0x150
> >> bpf_prog_fbdaf42eded9fe46_on_event+0x5e3/0x1000
> >> bpf_overflow_handler+0x60/0x100
> >> __perf_event_overflow+0x4f/0xf0
> >> perf_swevent_overflow+0x99/0xc0
> >> ___perf_sw_event+0xe7/0x120
> >> __schedule+0x47d/0x620
> >> schedule+0x29/0x90
> >> futex_wait_queue_me+0xb9/0x110
> >> futex_wait+0x139/0x230
> >> do_futex+0x2ac/0xa50
> >> __x64_sys_futex+0x13c/0x180
> >> do_syscall_64+0x42/0x100
> >> entry_SYSCALL_64_after_hwframe+0x44/0xa9
> >>
> >
> >> diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c
> >> index 052580c33d26..3b278f6b0c3e 100644
> >> --- a/kernel/bpf/stackmap.c
> >> +++ b/kernel/bpf/stackmap.c
> >> @@ -287,7 +287,7 @@ static void stack_map_get_build_id_offset(struct bpf_stack_build_id *id_offs,
> >> bool irq_work_busy = false;
> >> struct stack_map_irq_work *work = NULL;
> >>
> >> - if (in_nmi()) {
> >> + if (in_nmi() || this_rq_is_locked()) {
> >> work = this_cpu_ptr(&up_read_work);
> >> if (work->irq_work.flags & IRQ_WORK_BUSY)
> >> /* cannot queue more up_read, fallback */
> >
> > This is horrific crap. Just say no to that get_build_id_offset()
> > trainwreck.
>
> this is not a helpful comment.
> What issues do you see with this approach?
It will still generate deadlocks if I place a tracepoint inside a lock
that nests inside rq->lock, and it won't ever be able to detect that.
Say do the very same thing on trace_hrtimer_start(), which is under
cpu_base->lock, which nests inside rq->lock. That should give you an
AB-BA.
tracepoints / perf-overflow should _never_ take locks.
All of stack_map_get_build_id_offset() is just disguisting games; I did
tell you guys how to do lockless vma lookups a few years ago -- and yes,
that is invasive core mm surgery. But this is just disguisting hacks for
not wanting to do it right.
Basically the only semi-sane thing to do with that trainwreck is
s/in_nmi()/true/ and pray.
On top of that I just hate buildids in general.
