On Mon, Oct 12, 2020 at 2:49 PM Yaniv Agman <yanivagman@xxxxxxxxx> wrote: > > בתאריך יום ב׳, 12 באוק׳ 2020 ב-23:03 מאת Andrii Nakryiko > <andrii.nakryiko@xxxxxxxxx>: > > > > On Fri, Oct 9, 2020 at 1:24 PM Yaniv Agman <yanivagman@xxxxxxxxx> wrote: > > > > > > בתאריך יום ו׳, 9 באוק׳ 2020 ב-22:53 מאת Andrii Nakryiko > > > <andrii.nakryiko@xxxxxxxxx>: > > > > > > > > On Fri, Oct 9, 2020 at 12:32 PM Yaniv Agman <yanivagman@xxxxxxxxx> wrote: > > > > > > > > > > בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:39 מאת Andrii Nakryiko > > > > > <andrii.nakryiko@xxxxxxxxx>: > > > > > > > > > > > > On Fri, Oct 9, 2020 at 11:33 AM Yaniv Agman <yanivagman@xxxxxxxxx> wrote: > > > > > > > > > > > > > > בתאריך יום ו׳, 9 באוק׳ 2020 ב-21:21 מאת Daniel Borkmann > > > > > > > <daniel@xxxxxxxxxxxxx>: > > > > > > > > > > > > > > > > On 10/9/20 8:09 PM, Yaniv Agman wrote: > > > > > > > > > בתאריך יום ו׳, 9 באוק׳ 2020 ב-20:39 מאת Daniel Borkmann > > > > > > > > > <daniel@xxxxxxxxxxxxx>: > > > > > > > > >> > > > > > > > > >> On 10/9/20 6:56 PM, Yaniv Agman wrote: > > > > > > > > >>> בתאריך יום ו׳, 9 באוק׳ 2020 ב-19:27 מאת Daniel Borkmann > > > > > > > > >>> <daniel@xxxxxxxxxxxxx>: > > > > > > > > >>>> > > > > > > > > >>>> [ Cc +Yonghong ] > > > > > > > > >>>> > > > > > > > > >>>> On 10/9/20 6:05 PM, Yaniv Agman wrote: > > > > > > > > >>>>> Pulling the latest changes of libbpf and compiling my application with it, > > > > > > > > >>>>> I see the following error: > > > > > > > > >>>>> > > > > > > > > >>>>> ../libbpf/src//root/usr/include/bpf/bpf_helpers.h:99:10: error: > > > > > > > > >>>>> unknown register name 'r0' in asm > > > > > > > > >>>>> : "r0", "r1", "r2", "r3", "r4", "r5"); > > > > > > > > >>>>> > > > > > > > > >>>>> The commit which introduced this change is: > > > > > > > > >>>>> 80c7838600d39891f274e2f7508b95a75e4227c1 > > > > > > > > >>>>> > > > > > > > > >>>>> I'm not sure if I'm doing something wrong (missing include?), or this > > > > > > > > >>>>> is a genuine error > > > > > > > > >>>> > > > > > > > > >>>> Seems like your clang/llvm version might be too old. > > > > > > > > >>> > > > > > > > > >>> I'm using clang 10.0.1 > > > > > > > > >> > > > > > > > > >> Ah, okay, I see. Would this diff do the trick for you? > > > > > > > > > > > > > > > > > > Yes! Now it compiles without any problems! > > > > > > > > > > > > > > > > Great, thx, I'll cook proper fix and check with clang6 as Yonghong mentioned. > > > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > Does this happen because I'm first compiling using "emit-llvm" and > > > > > > > then using llc? > > > > > > > > > > > > So this must be the reason, but I'll wait for Yonghong to confirm. > > > > > > > > > > > > > I wish I could use bpf target directly, but I'm then having problems > > > > > > > with includes of asm code (like pt_regs and other stuff) > > > > > > > > > > > > Are you developing for a 32-bit platform? Or what exactly is the > > > > > > problem? I've been trying to solve problems for 32-bit arches recently > > > > > > by making libbpf smarter, that relies on CO-RE though. Is CO-RE an > > > > > > option for you? > > > > > > > > > > > > > > > > Examples for the errors I'm getting: > > > > > /lib/modules/4.14.199-1-MANJARO/build/arch/x86/include/asm/atomic.h:177:9: > > > > > error: invalid output constraint '+q' in asm > > > > > return xadd(&v->counter, i); > > > > > ^ > > > > > /lib/modules/4.14.199-1-MANJARO/build/arch/x86/include/asm/cmpxchg.h:234:25: > > > > > note: expanded from macro 'xadd' > > > > > #define xadd(ptr, inc) __xadd((ptr), (inc), LOCK_PREFIX) > > > > > ... > > > > > > > > > > From What I understood, this is a known issue for tracing programs > > > > > (like the one I'm developing) > > > > > > > > We do have a bunch of selftests that use pt_regs and include, say, > > > > linux/ptrace.h header. I wonder why we are not seeing these problems. > > > > Selftests, btw, are also built with -emit-llvm and then piping output > > > > to llc. > > > > > > > > So.. there must be something else going on. It's hard to guess like > > > > this without seeing the code, but maybe -D__TARGET_ARCH_$(SRCARCH) > > > > during compilation could help, just as an idea. > > > > > > > > > > Adding -D__TARGET_ARCH_$(SRCARCH) doesn't seem to help. > > > If you are interested in having a look at the code, > > > The code (event_monitor_ebpf.c) including the makefile is here: > > > https://github.com/yanivagman/tracee/tree/move_to_libbpf/tracee > > > > You are including a lot of kernel headers in that code. If any of them > > pulls in some x86-specific stuff, you would get this problem. Try to > > minimize the amount of includes you have there, maybe the issue will > > go away. > > > > I only added an include when I needed it, so I'm not sure I can remove > any of them... > > > If not, one other way would be to generate vmlinux.h for your specific > > kernel build. It will have memory layout of all the structs identical > > to what you get from kernel headers. Then you can replace all those > > includes with a single #include "vmlinux.h". You can disable CO-RE > > part with #define BPF_NO_PRESERVE_ACCESS_INDEX before vmlinux.h is > > included. > > > > My application is supposed to run on every linux environment with kernel > 4.14. > The only requirement from the user is having clang and kernel headers. > Is there a way to generate vmlinux.h from the given kernel headers? nope > From my understanding, vmlinux.h can only be generated from vmlinux > using bpftool, > but I don't want to require the users to supply vmlinux as well. you need either a vmlinux image or just .BTF binary section contents, which you can extract with objdump. But either way, binary BTF data is needed, bpftool won't parse/compile C headers to generate vmlinux.h. > > > But I'm a bit puzzled how your approach is going to work without CO-RE > > if you ever intend to run your program on more than one build of the > > kernel. Different kernel builds/versions might have different memory > > layouts of the structs you are relying on, so you'd need to compile > > them for each kernel build separately. So it's either CO-RE or > > on-the-host compilation (what BCC is doing). And if you are moving > > away from BCC, but not adopting CO-RE, I'm not sure how you are going > > to deal with that issue. > > > > The idea is to compile the code once on the target node, then use the > created object file in subsequent runs. > Unlike bcc, which compiles the bpf code on every run, > with this approach, we "install" the application once, then the > program should run immediately without any further required > compilation I see, so there is on-the-box compilation involved. If you can make sure you have vmlinux with DWARF on each target box, you can generate BTF on your own with pahole, I suppose. But vmlinux with DWARF becomes another requirement that you might not be able to satisfy. > The end goal is to support CO-RE, after most distros will support it > out of the box, and the user won't have to supply vmlinux for the > program to work > > > > > > > > > This is still WIP (the move to libbpf), and libbpf should be added as > > > a submodule to the project root > > > > > > > > Unfortunately, CO-RE is not (yet) an option. > > > > > I'm currently making the move from bcc to libbpf, and our application > > > > > needs to support kernel 4.14, and work on all environments. > > > > > > > > Kernel version is not a big problem, it's vmlinux BTF availability > > > > that could be a problem. vmlinux BTF can be added into any version of > > > > kernel with pahole -J, post factum, but that assumes you have some > > > > control over how kernels are built and distributed, of course. > > > > > > > > > > > > > > > [...]
