On 20/07/2023 22:08, Yan Zhai wrote: > On Thu, Jul 20, 2023 at 12:59 PM Alan Maguire <alan.maguire@xxxxxxxxxx> wrote: >> >> On 20/07/2023 18:40, Ivan Babrou wrote: >>> Hello, >>> >>> I noticed that CAP_SYS_ADMIN is required to attach BTF enabled probes >>> for modules. Attaching them for compiled-in points works just fine >>> without it. >>> >>> The reason is that libbpf calls into bpf_obj_get_next_id: >>> >>> #0 bpf_obj_get_next_id (start_id=start_id@entry=0, >>> next_id=next_id@entry=0x7fffcbffe578, cmd=cmd@entry=23) at bpf.c:908 >>> #1 0x00000000008bc08a in bpf_btf_get_next_id >>> (start_id=start_id@entry=0, next_id=next_id@entry=0x7fffcbffe578) at >>> bpf.c:930 >>> #2 0x00000000008ca252 in load_module_btfs >>> (obj=obj@entry=0x7fffc4004a40) at libbpf.c:5365 >>> #3 0x00000000008ca508 in find_kernel_btf_id >>> (btf_type_id=0x7fffcbffe73c, btf_obj_fd=0x7fffcbffe738, >>> attach_type=BPF_TRACE_FENTRY, attach_name=0xf8b647 >>> "nfnetlink_rcv_msg", obj=0x7fffc4004a40) at libbpf.c:9057 >>> #4 find_kernel_btf_id (obj=0x7fffc4004a40, attach_name=0xf8b647 >>> "nfnetlink_rcv_msg", attach_type=BPF_TRACE_FENTRY, >>> btf_obj_fd=0x7fffcbffe738, btf_type_id=0x7fffcbffe73c) at >>> libbpf.c:9042 >>> #5 0x00000000008ca755 in libbpf_find_attach_btf_id >>> (btf_type_id=0x7fffcbffe73c, btf_obj_fd=0x7fffcbffe738, >>> attach_name=0xf8b647 "nfnetlink_rcv_msg", prog=0x7fffc401d5b0) at >>> libbpf.c:9109 >>> #6 libbpf_prepare_prog_load (prog=0x7fffc401d5b0, >>> opts=0x7fffcbffe7c0, cookie=<optimized out>) at libbpf.c:6668 >>> #7 0x00000000008c3eb5 in bpf_object_load_prog >>> (obj=obj@entry=0x7fffc4004a40, prog=prog@entry=0x7fffc401d5b0, >>> insns=0x7fffc400ccc0, insns_cnt=87, >>> license=license@entry=0x7fffc4004a50 "GPL", >>> kern_version=<optimized out>, prog_fd=0x7fffc401d628) at libbpf.c:6741 >>> #8 0x00000000008d0294 in bpf_object__load_progs (log_level=<optimized >>> out>, obj=<optimized out>) at libbpf.c:7085 >>> #9 bpf_object_load (extra_log_level=0, target_btf_path=0x0, >>> obj=<optimized out>) at libbpf.c:7656 >>> #10 bpf_object__load (obj=<optimized out>) at libbpf.c:7703 >>> #11 0x00000000008b90e7 in _cgo_58a414c63447_Cfunc_bpf_object__load >>> (v=0xc000237bd8) at cgo-gcc-prolog:1232 >>> #12 0x000000000046c224 in runtime.asmcgocall () at >>> /usr/local/go/src/runtime/asm_amd64.s:848 >>> #13 0x00007fffcbfff260 in ?? () >>> #14 0x000000000041020e in runtime.persistentalloc.func1 () at >>> /usr/local/go/src/runtime/malloc.go:1393 >>> #15 0x000000000046a3a9 in runtime.systemstack () at >>> /usr/local/go/src/runtime/asm_amd64.s:496 >>> #16 0x00007fffffffdf6f in ?? () >>> #17 0x0100000000000000 in ?? () >>> #18 0x0000000000800000 in >>> github.com/golang/protobuf/ptypes/timestamp.file_github_com_golang_protobuf_ptypes_timestamp_timestamp_proto_init >>> () >>> at /home/builder/go/pkg/mod/github.com/golang/protobuf@v1.5.2/ptypes/timestamp/timestamp.pb.go:57 >>> #19 0x0000000000000000 in ?? () >>> >>> Here it is in code, where it happens after vmlinux does not find the >>> requested id: >>> >>> * https://github.com/libbpf/libbpf/blob/v1.2.0/src/libbpf.c#L9219 >>> >>> And in turn bpf_obj_get_next_id requires CAP_SYS_ADMIN here: >>> >>> * https://elixir.bootlin.com/linux/v6.5-rc1/source/kernel/bpf/syscall.c#L3790 >>> >>> The requirement comes from commit 34ad558 ("bpf: Add >>> BPF_(PROG|MAP)_GET_NEXT_ID command") from v4.13: >>> >>> * https://github.com/torvalds/linux/commit/34ad558 >>> >>> There's also this in the commit message: It is currently limited to >>> CAP_SYS_ADMIN which we can consider to lift it in followup patches. >>> >>> Later in v5.4 commit 341dfcf ("btf: expose BTF info through sysfs") >>> exposed BTF info via sysfs: >>> >>> * https://github.com/torvalds/linux/commit/341dfcf >>> >>> This info is world readable and it doesn't require any special capabilities: >>> >>> static struct bin_attribute bin_attr_btf_vmlinux __ro_after_init = { >>> .attr = { .name = "vmlinux", .mode = 0444, }, >>> .read = btf_vmlinux_read, >>> }; >>> >>> $ ls -l /sys/kernel/btf/vmlinux >>> -r--r--r-- 1 root root 4438336 Jul 13 06:33 /sys/kernel/btf/vmlinux >>> >>> My question is then: do we still need CAP_SYS_ADMIN? Should it be >>> CAP_BPF / CAP_PERFMON (available since v5.8) or should we drop the >>> requirement completely, since we expose vmlinux btf without any >>> restrictions? >>> >>> I'm happy to submit a patch. >>> >> >> I think it would be possible to gather module BTF data via >> /sys/kernel/btf instead of via iterating through the BTF objects, which >> is where lack of CAP_SYS_ADMIN trips up. The only problem is you won't >> have the BTF id of the module (which you get from the object), but I >> don't currently see that being used anywhere in libbpf. I might be >> missing something though. >> > sysfs does not have BTF exported if required modules have not been > loaded into the kernel. Loading modules would require SYS_ADMIN. Will > that be a problem? > I'd presume the approach would match existing behaviour, which would mean iterating over the set of loaded modules, just via /sys/kernel/btf instead of via BTF object iteration. I don't think there should be a need to load any modules. Alan > >> Alan >> > >
