On Mon, Sep 28, 2020 at 1:08 PM Yaniv Agman <yanivagman@xxxxxxxxx> wrote: > > בתאריך יום ב׳, 28 בספט׳ 2020 ב-8:50 מאת Andrii Nakryiko > <andrii.nakryiko@xxxxxxxxx>: > > > > On Fri, Sep 25, 2020 at 4:58 PM Yaniv Agman <yanivagman@xxxxxxxxx> wrote: > > > > > > Hello, > > > > > > I'm developing a tool which is now based on BCC, and would like to > > > make the move to libbpf. > > > I need the tool to support a minimal kernel version 4.14, which > > > doesn't have CO-RE. > > > > You don't need kernel itself to support CO-RE, you just need that > > kernel to have BTF in it. If the kernel is too old to have > > CONFIG_DEBUG_INFO_BTF config, you can still add BTF by running `pahole > > -J <path-to-vmlinux-image>`, if that's at all an option for your > > setup. > > > > Thanks, I didn't know that > > > > > > > I have read bcc-to-libbpf-howto-guide, and looked at the libbpf-tools of bcc, > > > but both only deal with newer kernels, and I failed to change them to > > > run with a 4.14 kernel. > > > > > > Although some of the bpf samples in the kernel source don't use CO-RE, > > > they all use bpf_load.h, > > > and have dependencies on the tools dir, which I would like to avoid. > > > > Depending on what exactly you are trying to achieve with your BPF > > application, you might not need BPF CO-RE, and using libbpf without > > CO-RE would be enough for your needs. This would be the case if you > > don't need to access any of the kernel data structures (e.g., all sort > > of networking BPF apps: TC programs, cgroup sock progs, XDP). But if > > you need to do anything tracing related (e.g., looking at kernel's > > task_struct or any other internal structure), then you have no choice > > and you either have to do on-the-target-host runtime compilation (BCC > > way) or relocations (libbpf + BPF CO-RE). This is because of changing > > memory layout of kernel structures. > > > > So, unless you can compile one specific version of your BPF code for a > > one specific version of the kernel, you need either BCC or BPF CO-RE. > > > > I'm working on a tracing application > (https://github.com/aquasecurity/tracee) which now uses bcc. We now > require a minimal kernel version 4.14, and bcc, but eventually we > would like to support CO-RE. I thought that we could do the move in > two steps. First moving to libbpf and keeping the 4.14 minimal > requirement, then adding CO-RE support in the future. > In order to do that, I thought of changing bcc requirement to clang > requirement, and compile the program once during installation on the > target host. This way we get the added value of fast start time > without the need to compile every time the program starts (like bcc > does), plus having an easier move to CO-RE in the future. Right, pre-compiling on the target machine with host kernel headers should work. So just don't use any of CO-RE features (no CO-RE relocations, no vmlinux.h), and it should just work. > > A problem that I encountered with kernel 4.14 and libbpf was that when > using bpf_prog_load (If I remember correctly), it returned an error of > invalid argument (-22). Doing a small investigation I saw that it > happened when trying to create bpf maps with names. Indeed I saw that > libbpf API changed between kernel 4.14 and 4.15 and the function > bpf_create_map_node now takes map name as an argument. Is there a way > to workaround this with kernel 4.14 and still use map names in > userspace to refer to bpf maps with libbpf? So we do run a few simple tests loading BPF programs (using libbpf) on 4.9 kernel, so map name should definitely not be a problem at all (libbpf is smart about detecting what's not supported in kernel and omitting non-essential things). It might be because of bpf_prog_load itself, which was long deprecated and you shouldn't use it for real-world applications. Please either use BPF skeleton or bpf_object APIs. It should just work, but if it doesn't please report back. > > > > > > > I would appreciate it if someone can help with a simple working > > > example of using libbpf on 4.14 kernel, without having any > > > dependencies. Specifically, I'm looking for an example makefile, and > > > to know how to load my bpf code with libbpf. > > > > libbpf-tools's Makefile would still work. Just drop dependency on > > vmlinux.h and include system headers directly, if necessary (and if > > you considered implications of kernel memory layout changes). > > > > Thanks, I'll try that > > > > > > > Thanks, > > > Yaniv
