> > Hi all, > > > > I'm trying to replicate a bpf test in the kernel that calls a function > > defined in the kernel itself. > > Source code is here: > > https://github.com/torvalds/linux/blob/v5.15/tools/testing/selftests/bpf/progs/kfunc_call_test.c > > > > I think I have all dependencies: > > Running within a qemu VM (Ubuntu 18.04) > > Kernel v 5.15 compiled from scratch with configs from > > tools/bpf/bpftool/feature.c > > pahole v1.22 (1.24 has a reported bug that doesn't allow me to use it) > > libbpf v1.0 > > Installed bpf tool from 5.15 kernel directory at `tools/bpf` > > clang and llvm 15 > > > > The goal is to call `bpf_kfunc_call_test1`, which is defined in > > net/bpf/test_run.c. > > I have two BPF programs and neither works. The first one is as is from > > the kernel: > > > > #include "vmlinux.h" > > #include <bpf/bpf_helpers.h> > > > > extern __u64 bpf_kfunc_call_test1(struct sock *sk, __u32 a, __u64 b, > > __u32 c, __u64 d) __ksym; > > > > SEC("classifier") > > int kfunc_call_test1(struct __sk_buff *skb) > > { > > struct sock *sk = 0; > > __u64 a; > > a = bpf_kfunc_call_test1(sk, 1, 2, 3, 4); > > hi, > IIUC you are passing 'sk' pointer defined on the stack, while > bpf_kfunc_call_test1 expects kernel pointer > > the kernel selftest test takes it from the skb with: > > struct bpf_sock *sk = skb->sk; I see. So even if the kernel function (bpf_kfunc_call_test1) does not use the argument, bpf is checking if it's a kernel pointer? Is the bpf compiler doing this check? I assumed that passing a constant 0 pointer would work since the other parameters are just constants, even in the kernel test. After changing the first program to the original code, the error changed, so that's progress. Now it says, even when running with root permissions: "libbpf: prog 'kfunc_call_test1': BPF program load failed: Permission denied" Would be interesting to know why, but not necessary since I won't use it this way. > > bpf_printk("bpf_kfunc_call_test1: %d.\n", a); > > return a; > > } > > > > > > It is compiled with these commands: > > > > bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h > > clang -g -O2 -target bpf -D__TARGET_ARCH_x86 -I. -idirafter > > /usr/lib/llvm-15/lib/clang/15.0.2/include -idirafter > > /usr/local/include -idirafter /usr/include/x86_64-linux-gnu -idirafter > > /usr/include -c hello.bpf.c -o hello.bpf.o > > llvm-strip -g hello.bpf.o > > bpftool gen skeleton hello.bpf.o > hello.skel.h > > cc -g -Wall hello.skel.h hello.c /usr/lib64/libbpf.a -lelf -lz -o hello > > > > > > The output is quite large, here is a gist: > > https://gist.github.com/hfingler/dc96af45d87004d0dc412e35be31709c. > > Mainly: > > > > libbpf: extern (func ksym) 'bpf_kfunc_call_test1': resolved to kernel [104983] > > libbpf: prog 'kfunc_call_test1': BPF program load failed: Invalid argument > > ... > > kernel function bpf_kfunc_call_test1 args#0 expected pointer to STRUCT > > sock but R1 is not a pointer to btf_id > > processed 6 insns (limit 1000000) max_states_per_insn 0 total_states 0 > > peak_states 0 mark_read 0 > > -- END PROG LOAD LOG -- > > libbpf: prog 'kfunc_call_test1': failed to load: -22 > > libbpf: failed to load object 'hello_bpf' > > libbpf: failed to load BPF skeleton 'hello_bpf': -22 > > Failed to load and verify BPF skeleton > > > > > > The other program is based off of minimal.c from libbpf-bootstrap. > > > > #include "vmlinux.h" > > #include <bpf/bpf_helpers.h> > > > > extern __u64 bpf_kfunc_call_test1(struct sock *sk, __u32 a, __u64 b, > > __u32 c, __u64 d) __ksym; > > > > SEC("tp/raw_syscalls/sys_enter") > > int handle_tp(void *ctx) > > { > > __u64 a; > > a = bpf_kfunc_call_test1(0, 1, 2, 3, 4); > > you can't call bpf_kfunc_call_test1 from tracepoint, it's registered for: > > ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_SCHED_CLS, &bpf_prog_test_kfunc_set); > ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING, &bpf_prog_test_kfunc_set); > ret = ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL, &bpf_prog_test_kfunc_set); > > check net/bpf/test_run.c > > also you're passing wrong arguments So there is some filtering happening. I'm on version 5.15 so I don't see that code, but I do see it on master. I'll try to figure out what is the matching code in 5.15 (or most likely just go to 5.19) and perhaps add BPF_PROG_TYPE_TRACEPOINT or whatever other PROG_TYPE I end up using. I'm assuming the wrong argument here is the same as above, that the argument must be a pointer in kernel space. I'll create a new bpf_kfunc_call_test1 that doesn't have a sock* argument, just ints and see if it works. > > bpf_printk("bpf_kfunc_call_test1: %d.\n", a); > > return 0; > > } > > > > > > The output is a little different, gist: > > https://gist.github.com/hfingler/ac69e286f9e527dfd678ef2d768e757c > > mainly: > > > > libbpf: extern (func ksym) 'bpf_kfunc_call_test1': resolved to kernel [104983] > > ... > > calling kernel function bpf_kfunc_call_test1 is not allowed > > libbpf: prog 'handle_tp': failed to load: -13 > > libbpf: failed to load object 'hello_bpf' > > libbpf: failed to load BPF skeleton 'hello_bpf': -13 > > Failed to load and verify BPF skeleton > > > > > > What could be the problem here? I'm mostly interested in the second > > program, so that I can use it on my own tracepoints and other places. > > > > I'm aware of filtering in net/core/filter.c, but I can't find any > > reference to `bpf_kfunc` functions. In fact, I added this to filter.c, > > where both functions just return true (I'm not concerned about > > security, this is just research): > > > > const struct bpf_verifier_ops my_verifier_ops = { > > .check_kfunc_call = export_the_world, > > .is_valid_access = accept_the_world, > > }; > > > > > > I'm assuming something is not allowing this program to call it, maybe > > it's the section it's put in. The kernel test's SEC is `classifier`, > > which is > > defined at tools/lib/bpf/libbpf.c as `BPF_PROG_SEC("classifier", > > BPF_PROG_TYPE_SCHED_CLS),`, while `tp/` is BPF_PROG_TYPE_TRACEPOINT. > > Is there a filter somewhere that allows one but not the other? For > > example, in kernel/bpf/syscall.c I see: > > > > static bool is_net_admin_prog_type(enum bpf_prog_type prog_type) > > { > > switch (prog_type) { > > case BPF_PROG_TYPE_SCHED_CLS: > > .. others > > > > but BPF_PROG_TYPE_TRACEPOINT is not here. > > > > There are so many references to these things that I'm totally lost, > > I'd appreciate some help. > > Artem recently added simple kfunc call for kexec: > 133790596406 bpf: export crash_kexec() as destructive kfunc > > might be easier way into kfuncs > > jirka Is creating a `struct bpf_verifier_ops` necessary? That patch you mentioned does not do it, but perhaps it's somewhere else or it's completely handled already. I currently have one that just returns true for both `check_kfunc_call` and `is_valid_access` but I'm not sure if it is necessary or how the "macro magic" for these work. The current kfunc documentation mentions we need to use `register_btf_kfunc_id_set` explicitly, something that isn't in 5.15. Is just registering the kfunc_id_set enough that I don't have to worry about the verifier_ops? I think I will use just kprobes and kfuncs. Thanks for the help!
