> > > > [...] > > > > As the target kernel does not support CONFIG_DEBUG_INFO_BTF, I used pahole -J (v1.22) to create vmlinux file with BTF info embedded there. > > > > Basically, I followed this mails: > > > > https://urldefense.com/v3/__https://lore.kernel.org/bpf/CADmGQ*1euj7Uv > > > > 9e8UyZMMXDiYAKqXe9=GSTBFNbbg1E0R-ejyg@xxxxxxxxxxxxxx/__;Kw!!MeVeBiz6!5 > > > > PZT7QBM-W93AhbZnRJ3kmTO4JyBUiapxeJrPCl4k4gKHidI5Ri0WQp19MbBDFP1nWOL3A$ > > > > > > > > Right now, the bpf program is just a uProbe for a simple test app, which writes some output to the tracing pipe. As Kernel 4.18. does not support global data for bpf programs, I had to remove (comment out) the bpf_trace_printk statements. > > > > > > You can do #define BPF_NO_GLOBAL_DATA before including bpf_helpers.h and then you can still use bpf_printk() helper macro. > > > > > > > Thank you! That's really helpful. Is there any collection of up-to-date documentation and best practices in writing bpf code (without skeletons) besides the sources in libbpf/libbpf-bootstrap, iovisor /libbpf-tools and linux/tools/testing/selftests/bpf ? > > There are also various blog posts on the Internet. I tried to do my > part with CO-RE at [0], but I don't think there is any single > documentation that captures all possible issues in one place (and I > don't think it's possible, there are just way too many variables to > anticipate all possible problems). > > [0] https://nakryiko.com/posts/bpf-core-reference-guide/ What a pity. ok thanks! > > > > > > > > > > > On the development machine, it works fine. But on the target machine, loading the program fails: libbpf: load bpf program failed: Invalid argument (full libbpf log see below). > > > > When compiling the programs on the target machine without using CO:RE, I get a similar error (invalid argument, -22). > > > > What could be the problem? I don't think the eBPF program uses anything that is available on Kernel 5.4.0 and not available on the system with Kernel 4.18, does it? > > > > > > > > Thanks in advance for your help. > > > > Best > > > > Dennis > > > > > > > > > > > > > > > > > > > > ============ log ============ > > > > > > > > sudo ./ebpf > > > > libbpf: loading main.bpf.o > > > > [...] > > > > libbpf: CO-RE relocating [0] struct pt_regs: found target candidate > > > > [201] struct pt_regs in [vmlinux] > > > > libbpf: prog 'trace_func_entry': relo #0: kind <byte_off> (0), spec is > > > > [2] struct pt_regs.di (0:14 @ offset 112) > > > > libbpf: prog 'trace_func_entry': relo #0: matching candidate #0 [201] > > > > struct pt_regs.di (0:14 @ offset 112) > > > > libbpf: prog 'trace_func_entry': relo #0: patched insn #2 (ALU/ALU64) > > > > imm 112 -> 112 > > > > libbpf: prog 'trace_func_entry': relo #1: kind <byte_off> (0), spec is > > > > [2] struct pt_regs.si (0:13 @ offset 104) > > > > libbpf: prog 'trace_func_entry': relo #1: matching candidate #0 [201] > > > > struct pt_regs.si (0:13 @ offset 104) > > > > libbpf: prog 'trace_func_entry': relo #1: patched insn #9 (ALU/ALU64) > > > > imm 104 -> 104 > > > > libbpf: sec 'kretprobe/': found 1 CO-RE relocations > > > > libbpf: prog 'trace_func_exit': relo #0: kind <byte_off> (0), spec is > > > > [2] struct pt_regs.ax (0:10 @ offset 80) > > > > libbpf: prog 'trace_func_exit': relo #0: matching candidate #0 [201] > > > > struct pt_regs.ax (0:10 @ offset 80) > > > > libbpf: prog 'trace_func_exit': relo #0: patched insn #2 (ALU/ALU64) > > > > imm 80 -> 80 > > > > > > CO-RE relocations succeeded, btf_custom_path worked, the problem is not in CO-RE. > > > > > > > libbpf: load bpf program failed: Invalid argument > > > > libbpf: failed to load program 'trace_func_entry' > > > > > > I suspect this is due to your target machine running Ubuntu 18.10. > > > Ubuntu has infamous problem with reporting kernel version through > > > uname() syscall. I've just improved libbpf's detection of it few days ago (see [0]), but it didn't yet make it into Github mirror of libbpf. > > > I'm going to start the sync right now, but you can manually specify correct version code with bpf_object__set_kversion() as you already realized. See how I do that in my patch [0], you can do that manually as well. > > > > > > [0] https://urldefense.com/v3/__https://patchwork.kernel.org/project/netdevbpf/patch/20211222231003.2334940-1-andrii@xxxxxxxxxx/__;!!MeVeBiz6!5PZT7QBM-W93AhbZnRJ3kmTO4JyBUiapxeJrPCl4k4gKHidI5Ri0WQp19MbBDFM_zuNNKA$ > > > > That's it, thank you very much! > > > > I just added > > __u32 currKernelVersion = get_kernel_version(); > > bpf_object__set_kversion(main_bpf_obj, currKernelVersion) > > with get_kernel_version() from libbpf.c and your patch [0] and now it is working. :) > > > > Great, once [1] is merged (I'll need to fix up selftests first), > libbpf will be able to deal with this Ubuntu quirk automatically and > you'll never know it existed. > > [1] https://github.com/libbpf/libbpf/pull/435 > > > However, there is something I do not understand: > > (a): Why did it work on the development machine in the first place with Ubuntu 20.04? > > - the old get_kernel_version() returned 328704 (5.4.0), the new version returns 328852 (5.4.148) so there already is a missmatch. (and I did not call it, so libbpf set the kernel version to 0?) > > You dev machine has recent enough kernel that doesn't check kernel > version anymore, so it doesn't matter that Ubuntu reports the wrong > one. > > > - on the target machine, it is 266752 (4.2.0) vs. 266772 (4.2.4) > > (b): I use a uProbe. Why is the bpf program type kProbe? Is it just for usability of libbpf as for a uProbe the user must specify the address while for a kprobe only the symbol is required?> > 4.2 is old enough where kernel version has to match and it didn't. > Thank you! > > (c): Why does the kernel version matter at all? > > > > It would be really nice if you could answer these questions as well or point me to a source. > > > > Best > > Dennis > > > > [...] > > >
