On 9/28/20 5:00 PM, Yaniv Agman wrote:
Hi Andrii,
I used BPF skeleton as you suggested, which did work with kernel 4.19
but not with 4.14.
I used the exact same program, same environment, only changed the
kernel version.
The error message I get on 4.14:
libbpf: elf: skipping unrecognized data section(5) .rodata.str1.1
libbpf: failed to determine kprobe perf type: No such file or directory
libbpf: prog 'kprobe__do_sys_open': failed to create kprobe
'do_sys_open' perf event: No such file or directory
libbpf: failed to auto-attach program 'kprobe__do_sys_open': -2
failed to attach BPF programs: No such file or directory
As the program I made is small, I'm copying it here:
===========================================
#include <linux/version.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include <uapi/linux/bpf.h>
#include <uapi/linux/ptrace.h>
struct bpf_map_def SEC("maps") open_fds = {
.type = BPF_MAP_TYPE_LRU_HASH,
.key_size = sizeof(int),
.value_size = sizeof(int),
.max_entries = 1024,
};
SEC("kprobe/do_sys_open")
int BPF_KPROBE(kprobe__do_sys_open)
{
int err;
u32 id = bpf_get_current_pid_tgid();
int dfd = PT_REGS_PARM1(ctx);
if ((err = bpf_map_update_elem(&open_fds, &id, &dfd, BPF_ANY))) {
char log[] = "bpf_map_update_elem %d\n";
put the above definition as global like
const char log[] = "bpf_map_update_elem %d\n";
might help.
bpf_trace_printk(log, sizeof(log), err);
return 1;
}
return 0;
}
char LICENSE[] SEC("license") = "GPL";
==================================================
Can you think of a reason why this only happens on 4.14?
Thanks,
Yaniv
בתאריך יום ב׳, 28 בספט׳ 2020 ב-23:24 מאת Andrii Nakryiko
<andrii.nakryiko@xxxxxxxxx>:
On Mon, Sep 28, 2020 at 1:08 PM Yaniv Agman <yanivagman@xxxxxxxxx> wrote:
בתאריך יום ב׳, 28 בספט׳ 2020 ב-8:50 מאת Andrii Nakryiko
<andrii.nakryiko@xxxxxxxxx>:
On Fri, Sep 25, 2020 at 4:58 PM Yaniv Agman <yanivagman@xxxxxxxxx> wrote:
Hello,
I'm developing a tool which is now based on BCC, and would like to
make the move to libbpf.
I need the tool to support a minimal kernel version 4.14, which
doesn't have CO-RE.
You don't need kernel itself to support CO-RE, you just need that
kernel to have BTF in it. If the kernel is too old to have
CONFIG_DEBUG_INFO_BTF config, you can still add BTF by running `pahole
-J <path-to-vmlinux-image>`, if that's at all an option for your
setup.
Thanks, I didn't know that
I have read bcc-to-libbpf-howto-guide, and looked at the libbpf-tools of bcc,
but both only deal with newer kernels, and I failed to change them to
run with a 4.14 kernel.
Although some of the bpf samples in the kernel source don't use CO-RE,
they all use bpf_load.h,
and have dependencies on the tools dir, which I would like to avoid.
Depending on what exactly you are trying to achieve with your BPF
application, you might not need BPF CO-RE, and using libbpf without
CO-RE would be enough for your needs. This would be the case if you
don't need to access any of the kernel data structures (e.g., all sort
of networking BPF apps: TC programs, cgroup sock progs, XDP). But if
you need to do anything tracing related (e.g., looking at kernel's
task_struct or any other internal structure), then you have no choice
and you either have to do on-the-target-host runtime compilation (BCC
way) or relocations (libbpf + BPF CO-RE). This is because of changing
memory layout of kernel structures.
So, unless you can compile one specific version of your BPF code for a
one specific version of the kernel, you need either BCC or BPF CO-RE.
I'm working on a tracing application
(https://github.com/aquasecurity/tracee) which now uses bcc. We now
require a minimal kernel version 4.14, and bcc, but eventually we
would like to support CO-RE. I thought that we could do the move in
two steps. First moving to libbpf and keeping the 4.14 minimal
requirement, then adding CO-RE support in the future.
In order to do that, I thought of changing bcc requirement to clang
requirement, and compile the program once during installation on the
target host. This way we get the added value of fast start time
without the need to compile every time the program starts (like bcc
does), plus having an easier move to CO-RE in the future.
Right, pre-compiling on the target machine with host kernel headers
should work. So just don't use any of CO-RE features (no CO-RE
relocations, no vmlinux.h), and it should just work.
A problem that I encountered with kernel 4.14 and libbpf was that when
using bpf_prog_load (If I remember correctly), it returned an error of
invalid argument (-22). Doing a small investigation I saw that it
happened when trying to create bpf maps with names. Indeed I saw that
libbpf API changed between kernel 4.14 and 4.15 and the function
bpf_create_map_node now takes map name as an argument. Is there a way
to workaround this with kernel 4.14 and still use map names in
userspace to refer to bpf maps with libbpf?
So we do run a few simple tests loading BPF programs (using libbpf) on
4.9 kernel, so map name should definitely not be a problem at all
(libbpf is smart about detecting what's not supported in kernel and
omitting non-essential things). It might be because of bpf_prog_load
itself, which was long deprecated and you shouldn't use it for
real-world applications. Please either use BPF skeleton or bpf_object
APIs. It should just work, but if it doesn't please report back.
I would appreciate it if someone can help with a simple working
example of using libbpf on 4.14 kernel, without having any
dependencies. Specifically, I'm looking for an example makefile, and
to know how to load my bpf code with libbpf.
libbpf-tools's Makefile would still work. Just drop dependency on
vmlinux.h and include system headers directly, if necessary (and if
you considered implications of kernel memory layout changes).
Thanks, I'll try that
Thanks,
Yaniv
