On Wed, Apr 26, 2023 at 5:14 PM Namhyung Kim <namhyung@xxxxxxxxxx> wrote:
>
> Hello,
>
> I'm having a problem of loading perf lock contention BPF program [1]
> on old kernels. It has collect_lock_syms() to get the address of each
> CPU's run-queue lock. The kernel 5.14 changed the name of the field
> so there's bpf_core_field_exists to check the name like below.
>
> if (bpf_core_field_exists(rq_new->__lock))
> lock_addr = (__u64)&rq_new->__lock;
> else
> lock_addr = (__u64)&rq_old->lock;
I suspect compiler rewrites it to something like
lock_addr = (__u64)&rq_old->lock;
if (bpf_core_field_exists(rq_new->__lock))
lock_addr = (__u64)&rq_new->__lock;
so rq_old relocation always happens and ends up being not guarded
properly. You can try adding barrier_var(rq_new) and
barrier_var(rq_old) around if and inside branches, that should
pessimize compiler
alternatively if you do
if (bpf_core_field_exists(rq_new->__lock))
lock_addr = (__u64)&rq_new->__lock;
else if (bpf_core_field_exists(rq_old->lock))
lock_addr = (__u64)&rq_old->lock;
else
lock_addr = 0; /* or signal error somehow */
It might work as well.
>
> Note that I've applied a patch [2] to fix an issue with this code.
>
> It works fine on my machine (with a newer kernel), but failed on the
> old kernels. I guess it'd go to the else part without a problem but
> it didn't for some reason.
>
> Then I change the code to check the rq_old first. It works well on
> the old kernels but fails on newer kernels.. :(
>
> libbpf: prog 'collect_lock_syms': BPF program load failed: Invalid argument
> libbpf: prog 'collect_lock_syms': -- BEGIN PROG LOAD LOG --
> reg type unsupported for arg#0 function collect_lock_syms#380
> 0: R1=ctx(off=0,imm=0) R10=fp0
> ; int BPF_PROG(collect_lock_syms)
> 0: (b7) r6 = 0 ; R6_w=0
> 1: (b7) r7 = 0 ; R7_w=0
> 2: (b7) r9 = 1 ; R9_w=1
> 3: <invalid CO-RE relocation>
> failed to resolve CO-RE relocation <byte_off> [381] struct rq___old.lock (0:0 @ offset 0)
> processed 4 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0
>
> I'm curious what went wrong with this. I guess it's supposed to work
> on any kernel verions by definition. Not sure the compiler generated
> a wrong reloc or something. Maybe I just made silly mistakes..
>
> Do you see anything wrong? Any hints to debug this issue?
>
> Thanks,
> Namhyung
>
>
> [1] file://linux/tools/perf/util/bpf_skel/lock_contention.bpf.c
> [2] https://lore.kernel.org/lkml/20230423215650.287812-1-namhyung@xxxxxxxxxx/
