> > [...] > > As the target kernel does not support CONFIG_DEBUG_INFO_BTF, I used pahole -J (v1.22) to create vmlinux file with BTF info embedded there. > > Basically, I followed this mails: > > https://urldefense.com/v3/__https://lore.kernel.org/bpf/CADmGQ*1euj7Uv > > 9e8UyZMMXDiYAKqXe9=GSTBFNbbg1E0R-ejyg@xxxxxxxxxxxxxx/__;Kw!!MeVeBiz6!5 > > PZT7QBM-W93AhbZnRJ3kmTO4JyBUiapxeJrPCl4k4gKHidI5Ri0WQp19MbBDFP1nWOL3A$ > > > > Right now, the bpf program is just a uProbe for a simple test app, which writes some output to the tracing pipe. As Kernel 4.18. does not support global data for bpf programs, I had to remove (comment out) the bpf_trace_printk statements. > > You can do #define BPF_NO_GLOBAL_DATA before including bpf_helpers.h and then you can still use bpf_printk() helper macro. > Thank you! That's really helpful. Is there any collection of up-to-date documentation and best practices in writing bpf code (without skeletons) besides the sources in libbpf/libbpf-bootstrap, iovisor /libbpf-tools and linux/tools/testing/selftests/bpf ? > > > On the development machine, it works fine. But on the target machine, loading the program fails: libbpf: load bpf program failed: Invalid argument (full libbpf log see below). > > When compiling the programs on the target machine without using CO:RE, I get a similar error (invalid argument, -22). > > What could be the problem? I don't think the eBPF program uses anything that is available on Kernel 5.4.0 and not available on the system with Kernel 4.18, does it? > > > > Thanks in advance for your help. > > Best > > Dennis > > > > > > > > > > ============ log ============ > > > > sudo ./ebpf > > libbpf: loading main.bpf.o > > [...] > > libbpf: CO-RE relocating [0] struct pt_regs: found target candidate > > [201] struct pt_regs in [vmlinux] > > libbpf: prog 'trace_func_entry': relo #0: kind <byte_off> (0), spec is > > [2] struct pt_regs.di (0:14 @ offset 112) > > libbpf: prog 'trace_func_entry': relo #0: matching candidate #0 [201] > > struct pt_regs.di (0:14 @ offset 112) > > libbpf: prog 'trace_func_entry': relo #0: patched insn #2 (ALU/ALU64) > > imm 112 -> 112 > > libbpf: prog 'trace_func_entry': relo #1: kind <byte_off> (0), spec is > > [2] struct pt_regs.si (0:13 @ offset 104) > > libbpf: prog 'trace_func_entry': relo #1: matching candidate #0 [201] > > struct pt_regs.si (0:13 @ offset 104) > > libbpf: prog 'trace_func_entry': relo #1: patched insn #9 (ALU/ALU64) > > imm 104 -> 104 > > libbpf: sec 'kretprobe/': found 1 CO-RE relocations > > libbpf: prog 'trace_func_exit': relo #0: kind <byte_off> (0), spec is > > [2] struct pt_regs.ax (0:10 @ offset 80) > > libbpf: prog 'trace_func_exit': relo #0: matching candidate #0 [201] > > struct pt_regs.ax (0:10 @ offset 80) > > libbpf: prog 'trace_func_exit': relo #0: patched insn #2 (ALU/ALU64) > > imm 80 -> 80 > > CO-RE relocations succeeded, btf_custom_path worked, the problem is not in CO-RE. > > > libbpf: load bpf program failed: Invalid argument > > libbpf: failed to load program 'trace_func_entry' > > I suspect this is due to your target machine running Ubuntu 18.10. > Ubuntu has infamous problem with reporting kernel version through > uname() syscall. I've just improved libbpf's detection of it few days ago (see [0]), but it didn't yet make it into Github mirror of libbpf. > I'm going to start the sync right now, but you can manually specify correct version code with bpf_object__set_kversion() as you already realized. See how I do that in my patch [0], you can do that manually as well. > > [0] https://urldefense.com/v3/__https://patchwork.kernel.org/project/netdevbpf/patch/20211222231003.2334940-1-andrii@xxxxxxxxxx/__;!!MeVeBiz6!5PZT7QBM-W93AhbZnRJ3kmTO4JyBUiapxeJrPCl4k4gKHidI5Ri0WQp19MbBDFM_zuNNKA$ That's it, thank you very much! I just added __u32 currKernelVersion = get_kernel_version(); bpf_object__set_kversion(main_bpf_obj, currKernelVersion) with get_kernel_version() from libbpf.c and your patch [0] and now it is working. :) However, there is something I do not understand: (a): Why did it work on the development machine in the first place with Ubuntu 20.04? - the old get_kernel_version() returned 328704 (5.4.0), the new version returns 328852 (5.4.148) so there already is a missmatch. (and I did not call it, so libbpf set the kernel version to 0?) - on the target machine, it is 266752 (4.2.0) vs. 266772 (4.2.4) (b): I use a uProbe. Why is the bpf program type kProbe? Is it just for usability of libbpf as for a uProbe the user must specify the address while for a kprobe only the symbol is required? (c): Why does the kernel version matter at all? It would be really nice if you could answer these questions as well or point me to a source. Best Dennis [...]
