On Wed, May 25, 2022 at 9:13 AM Matteo Nardi <matteo@xxxxxxxx> wrote:
>
> This program will compile and run fine on my 64 bit system, but it
> will fail with a relocation error on 32 bit systems:
>
> SEC("tp/raw_syscalls/sys_enter")
> int sys_enter(struct trace_event_raw_sys_enter *ctx) {
> long int n = ctx->id;
> bpf_printk("hello world %d", n);
> return 0;
> }
>
> ```
> libbpf: prog 'sys_enter': relo #0: insn #0 (LDX/ST/STX) accesses field
> incorrectly. Make sure you are accessing pointers, unsigned integers,
> or fields of matching type and size.
> libbpf: prog 'sys_enter': BPF program load failed: Invalid argument
> libbpf: prog 'sys_enter': -- BEGIN PROG LOAD LOG --
> R1 type=ctx expected=fp
> ; long int n = ctx->id;
> 0: (85) call unknown#195896080
> invalid func unknown#195896080
> processed 1 insns (limit 1000000) max_states_per_insn 0 total_states 0
> peak_states 0 mark_read 0
> -- END PROG LOAD LOG --
> libbpf: failed to load program 'sys_enter'
> libbpf: failed to load object 'bootstrap_bpf'
> libbpf: failed to load BPF skeleton 'bootstrap_bpf': -22
> Failed to load and verify BPF skeleton
> ```
>
> I'm cross-compiling using a Yocto build. I've reproduced this both
> with arm and x86.
>
> From my understanding, the issue comes from the `long int` in
> `trace_event_raw_sys_enter`, which is 64 bit in the compiled eBPF
> program, but 32 bit in the target kernel.
>
> struct trace_event_raw_sys_enter {
> struct trace_entry ent;
> long int id;
> long unsigned int args[6];
> char __data[0];
> } __attribute__((preserve_access_index));
>
> Indeed, manually changing the `id` definition in `vmlinux.h` will fix
> the relocation error:
>
> struct trace_event_raw_sys_enter {
> u32 id;
> } __attribute__((preserve_access_index));
>
>
> "Q: clang flag for target bpf?"[0] hints that using a native target
> could help, but I guess that would completely break CORE relocations
> since `preserve_access_index` is a `-target bpf`-specific attribute,
> right?
>
> Am I missing something? If I had to fix the issue right now I would
> replace all long definitions in `vmlinux.h` to u32 when targeting 32
> bit systems. Could `bpftool btf dump` handle this?
> We're using eBPF on embedded systems, where 32 bit is still fairly common.
>
This could work for long, but there is nothing that can be done for
pointers. Good knews is that libbpf can auto-matically adjust such 64
vs 32 bit accesses for pointers and *unsigned* integers, but you ran
into an issue with signed integer, which isn't as easy to handle due
to the need to do sign-extension.
Libbpf could handle this transparently pretty easily if BPF had
sign-extending load, though, so maybe the best long-term solution is
to have one? It actually would make some other cases better and allow
Clang to not generate a sequence of <<32, >>32 shift just for sign
extensions. Alexei, Yonghong, WDYT?
> Thanks.
>
> Best regards,
> Matteo Nardi
>
> [0] https://www.kernel.org/doc/html/latest/bpf/bpf_devel_QA.html#q-clang-flag-for-target-bpf
