On Fri, 23 Jun 2023 at 22:54, Andrii Nakryiko <andrii.nakryiko@xxxxxxxxx> wrote:
>
> On Wed, Jun 21, 2023 at 3:51 AM andrea terzolo <andreaterzolo3@xxxxxxxxx> wrote:
> >
> > Hi all!
> >
> > Recently I faced a strange issue with CO-RE relocations and the
> > required privileged to run our eBPF probe. In Falco, we try to support
> > a vast range of kernel versions and distros, so to support COS systems
>
> what's COS?
>
oh sorry for that! COS stands for "Container-Optimized OS" and is one
of the most used OS on GKE(Google Kubernetes Engine) nodes
> > we added this custom patch [0]. More in detail:
> >
> > if(...)
> > {
> > }
> > else
> > {
> > struct task_struct___cos *task_cos = (void *)task;
> >
> > if(bpf_core_field_exists(task_cos->audit->loginuid))
>
> unrelated to your issue, but I think you are misusing
> bpf_core_field_exists() here. You should only have one arrow in the
> field expression (i.e., no extra pointer dereferences except). Or
> better use the form bpf_core_field_exists(struct task_struct___cos,
> audit). As you wrote it, it might be checking only existence of
> loginuid inside typeof(task_cos->audit), but it doesn't check that
> task_struct has audit field.
>
thank you very much for the suggestion!
> > {
> > BPF_CORE_READ_INTO(loginuid, task_cos, audit, loginuid.val);
> > }
> > }
> >
> > The issue is that now when running on not-COS systems we face this
> > error when using only `CAP_BPF` and `CAP_PERFMON` capabilities:
> >
> > libbpf: failed to iterate BTF objects: -1
> > libbpf: prog 't1_execve_x': relo #791: target candidate search failed
> > for [1238] struct audit_task_info: -1
> > libbpf: prog 't1_execve_x': relo #791: failed to relocate: -1
> > libbpf: failed to perform CO-RE relocations: -1
> > libbpf: failed to load object 'bpf_probe'
> > libbpf: failed to load BPF skeleton 'bpf_probe': -1
> >
> > If we use CAP_SYS_ADMIN all seems to work fine. The issue seems
> > related to the fact that during the relocation libbpf is not able
> > to find `audit_task_info` in the running kernel BTF, since we are not
> > running on COS system, and for this reason, it searches for it in
> > modules BTF, but in order to do that we need CAP_SYS_ADMIN[1].
> > Is this the intended behavior?
>
> Not really, though it is unfortunate that we need CAP_SYS_ADMIN just
> to find kernel module's BTF. cc Alexei, maybe we can relax some rules
> at least for BTFs?
>
> > If we want to support specific kernel structs like `audit_task_info`
> > do we need to run with CAP_SYS_ADMIN always enabled?
> > Is there a way to disable BTF module search with libbpf?
>
> We should probably just say that if CAP_SYS_ADMIN is not granted, we
> can't relocate against kernel module BTFs.
>
> In load_module_btfs(), just add an extra check after
> bpf_btf_get_next_id() for -EPERM. Would you like to submit a fix?
>
Sure! I will submit it!
> >
> > Side point:
> > Not sure this is the right place to report it but it seems that some
> > COS versions ([2]) backported something wrong: they backported the
> > `BPF_FUNC_ktime_get_coarse_ns` bpf helper but not the memcg-based
> > memory accounting. For this reason, libbpf doesn't bump the
> > RLIMIT_MEMLOCK supposing that the system uses a memcg-based memory
> > accounting and so we face the same error reported here [3]
>
> this feature detection gap is a known issue, unfortunately. There is
> no nice way to detect the need for memcg-based accounting,
> unfortunately. You'll have to bump RLIMI_MEMLOCK yourself, sorry.
>
Sounds good, thank you!
>
> >
> > [0]: https://github.com/falcosecurity/libs/pull/1062
> > [1]: https://github.com/torvalds/linux/blob/692b7dc87ca6d55ab254f8259e6f970171dc9d01/kernel/bpf/syscall.c#L3704
> > [2]: https://github.com/falcosecurity/falco/issues/2626
> > [3]: https://lore.kernel.org/netdev/20220610112648.29695-1-quentin@xxxxxxxxxxxxx/T/
> >
