בתאריך יום ג׳, 29 בספט׳ 2020 ב-4:29 מאת Andrii Nakryiko
<andrii.nakryiko@xxxxxxxxx>:
>
> On Mon, Sep 28, 2020 at 5:01 PM Yaniv Agman <yanivagman@xxxxxxxxx> wrote:
> >
> > Hi Andrii,
> >
> > I used BPF skeleton as you suggested, which did work with kernel 4.19
> > but not with 4.14.
> > I used the exact same program, same environment, only changed the
> > kernel version.
> > The error message I get on 4.14:
> >
> > libbpf: elf: skipping unrecognized data section(5) .rodata.str1.1
> > libbpf: failed to determine kprobe perf type: No such file or directory
>
> This means that your kernel doesn't support attaching to
> kprobe/tracepoint through perf_event subsystem. That's currently the
> only way that libbpf supports for kprobe/tracapoint programs. It was
> added in 4.17 kernel, which explains what is happening in your case.
> It is still possible to attach to kprobe using legacy ways, but libbpf
> doesn't provide that out of the box. We had a discussion a while ago
> (about 1 year ago) about adding that to libbpf, but at that time we
> didn't have a good testing infrastructure to validate such legacy
> interfaces, plus it's a bit on the unsafe side as far as APIs go
> (there is no auto-detachment and cleanup with how old kernels allow to
> do kprobe/tracepoint). But we might reconsider, given it's not a first
> time I see people get confused and blocked by this.
>
> Anyways, here's how you can do it without waiting for libbpf to do
> this out of the box:
>
>
> int poke_kprobe_events(bool add, const char* name, bool ret) {
> char buf[256];
> int fd, err;
>
> fd = open("/sys/kernel/debug/tracing/kprobe_events", O_WRONLY | O_APPEND, 0);
> if (fd < 0) {
> err = -errno;
> fprintf(stderr, "failed to open kprobe_events file: %d\n", err);
> return err;
> }
>
> if (add)
> snprintf(buf, sizeof(buf), "%c:kprobes/%s %s", ret ? 'r' : 'p', name, name);
> else
> snprintf(buf, sizeof(buf), "-:kprobes/%s", name);
>
> err = write(fd, buf, strlen(buf));
> if (err < 0) {
> err = -errno;
> fprintf(
> stderr,
> "failed to %s kprobe '%s': %d\n",
> add ? "add" : "remove",
> buf,
> err);
> }
> close(fd);
> return err >= 0 ? 0 : err;
> }
>
> int add_kprobe_event(const char* func_name, bool is_kretprobe) {
> return poke_kprobe_events(true /*add*/, func_name, is_kretprobe);
> }
>
> int remove_kprobe_event(const char* func_name, bool is_kretprobe) {
> return poke_kprobe_events(false /*remove*/, func_name, is_kretprobe);
> }
>
> struct bpf_link* attach_kprobe_legacy(
> struct bpf_program* prog,
> const char* func_name,
> bool is_kretprobe) {
> char fname[256], buf[256];
> struct perf_event_attr attr;
> struct bpf_link* link;
> int fd = -1, err, id;
> FILE* f = NULL;
>
> err = add_kprobe_event(func_name, is_kretprobe);
> if (err) {
> fprintf(stderr, "failed to create kprobe event: %d\n", err);
> return NULL;
> }
>
> snprintf(
> fname,
> sizeof(fname),
> "/sys/kernel/debug/tracing/events/kprobes/%s/id",
> func_name);
> f = fopen(fname, "r");
> if (!f) {
> fprintf(stderr, "failed to open kprobe id file '%s': %d\n", fname, -errno);
> goto err_out;
> }
>
> if (fscanf(f, "%d\n", &id) != 1) {
> fprintf(stderr, "failed to read kprobe id from '%s': %d\n", fname, -errno);
> goto err_out;
> }
>
> fclose(f);
> f = NULL;
>
> memset(&attr, 0, sizeof(attr));
> attr.size = sizeof(attr);
> attr.config = id;
> attr.type = PERF_TYPE_TRACEPOINT;
> attr.sample_period = 1;
> attr.wakeup_events = 1;
>
> fd = syscall(__NR_perf_event_open, &attr, -1, 0, -1, PERF_FLAG_FD_CLOEXEC);
> if (fd < 0) {
> fprintf(
> stderr,
> "failed to create perf event for kprobe ID %d: %d\n",
> id,
> -errno);
> goto err_out;
> }
>
> link = bpf_program__attach_perf_event(prog, fd);
> err = libbpf_get_error(link);
> if (err) {
> fprintf(stderr, "failed to attach to perf event FD %d: %d\n", fd, err);
> goto err_out;
> }
>
> return link;
>
> err_out:
> if (f)
> fclose(f);
> if (fd >= 0)
> close(fd);
> remove_kprobe_event(func_name, is_kretprobe);
> return NULL;
> }
>
>
> Then you'd use it in your application as:
>
> ...
>
> skel->links.handler = attach_kprobe_legacy(
> skel->progs.handler, "do_sys_open", false /* is_kretprobe */);
> if (!skel->links.handler) {
> fprintf(stderr, "Failed to attach kprobe using legacy debugfs API!\n");
> err = 1;
> goto out;
> }
>
> ... kprobe is attached here ...
>
> out:
> /* first clean up step */
> bpf_link__destroy(skel->links.handler);
> /* this is second necessary clean up step */
> remove_kprobe_event("do_sys_open", false /* is_kretprobe */);
>
>
> Let me know if that worked.
>
Thanks Andrii,
I made a small change for the code to compile:
skel->links.handler to skel->links.kprobe__do_sys_open and same for skel->progs
After compiling the code, I'm now getting the following error:
failed to create perf event for kprobe ID 1930: -2
Failed to attach kprobe using legacy debugfs API!
failed to remove kprobe '-:kprobes/do_sys_open': -2
As our application is written in go,
I hoped libbpf would support kernel 3.14 out of the box, so we can
just call libbpf functions using cgo wrappers.
I can do further checks if you'd like, but I think we will also
consider updating the minimal kernel version requirement to 4.18
> > libbpf: prog 'kprobe__do_sys_open': failed to create kprobe
> > 'do_sys_open' perf event: No such file or directory
> > libbpf: failed to auto-attach program 'kprobe__do_sys_open': -2
> > failed to attach BPF programs: No such file or directory
> >
>
> [...]
