On Wed, Dec 20, 2023 at 07:04:16PM +0100, Dmitrii Dolgov wrote:
> Currently, it's not allowed to attach an fentry/fexit prog to another
> one fentry/fexit. At the same time it's not uncommon to see a tracing
> program with lots of logic in use, and the attachment limitation
> prevents usage of fentry/fexit for performance analysis (e.g. with
> "bpftool prog profile" command) in this case. An example could be
> falcosecurity libs project that uses tp_btf tracing programs.
>
> Following the corresponding discussion [1], the reason for that is to
> avoid tracing progs call cycles without introducing more complex
> solutions. But currently it seems impossible to load and attach tracing
> programs in a way that will form such a cycle. The limitation is coming
> from the fact that attach_prog_fd is specified at the prog load (thus
> making it impossible to attach to a program loaded after it in this
> way), as well as tracing progs not implementing link_detach.
>
> Replace "no same type" requirement with verification that no more than
> one level of attachment nesting is allowed. In this way only one
> fentry/fexit program could be attached to another fentry/fexit to cover
> profiling use case, and still no cycle could be formed. To implement,
> add a new field into bpf_prog_aux to track nested attachment for tracing
> programs.
SNIP
> diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> index eb447b0a9423..e7393674ab94 100644
> --- a/include/linux/bpf.h
> +++ b/include/linux/bpf.h
> @@ -1414,6 +1414,7 @@ struct bpf_prog_aux {
> bool dev_bound; /* Program is bound to the netdev. */
> bool offload_requested; /* Program is bound and offloaded to the netdev. */
> bool attach_btf_trace; /* true if attaching to BTF-enabled raw tp */
> + bool attach_tracing_prog; /* true if tracing another tracing program */
> bool func_proto_unreliable;
> bool sleepable;
> bool tail_call_reachable;
> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
> index 5e43ddd1b83f..c40cad8886e9 100644
> --- a/kernel/bpf/syscall.c
> +++ b/kernel/bpf/syscall.c
> @@ -2702,6 +2702,22 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr, u32 uattr_size)
> goto free_prog_sec;
> }
>
> + /*
> + * Bookkeeping for managing the program attachment chain.
> + *
> + * It might be tempting to set attach_tracing_prog flag at the attachment
> + * time, but this will not prevent from loading bunch of tracing prog
> + * first, then attach them one to another.
hi,
sorry for delayed response.. this part gets trickier with every change :-)
> + *
> + * The flag attach_tracing_prog is set for the whole program lifecycle, and
> + * doesn't have to be cleared in bpf_tracing_link_release, since tracing
> + * programs cannot change attachment target.
I'm not sure that's the case.. AFAICS the bpf_tracing_prog_attach can
be called on already loaded program with different target program it
was loaded for, like:
load fentry1 -> bpf_test_fentry1
load fentry2 -> fentry1
fentry2->attach_tracing_prog = true
load ext1 -> prog
attach fentry2 -> ext1
in which case we drop the tgt_prog from loading time
and attach fentry2 to ext1
but I think we could just fix with resseting the attach_tracing_prog
in bpf_tracing_prog_attach when the tgt_prog switch happens
it'd be great to have test for that.. also to find out it's real case,
I'm not sure I haven't overlooked anything
jirka
> + */
> + if (type == BPF_PROG_TYPE_TRACING && dst_prog &&
> + dst_prog->type == BPF_PROG_TYPE_TRACING) {
> + prog->aux->attach_tracing_prog = true;
> + }
> +
> /* find program type: socket_filter vs tracing_filter */
> err = find_prog_type(type, prog);
> if (err < 0)
> @@ -3135,7 +3151,12 @@ static int bpf_tracing_prog_attach(struct bpf_prog *prog,
> }
>
> if (tgt_prog_fd) {
> - /* For now we only allow new targets for BPF_PROG_TYPE_EXT */
> + /*
> + * For now we only allow new targets for BPF_PROG_TYPE_EXT. If this
> + * part would be changed to implement the same for
> + * BPF_PROG_TYPE_TRACING, do not forget to update the way how
> + * attach_tracing_prog flag is set.
> + */
> if (prog->type != BPF_PROG_TYPE_EXT) {
> err = -EINVAL;
> goto out_put_prog;
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index 8e7b6072e3f4..f8c15ce8fd05 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -20077,6 +20077,7 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
> struct bpf_attach_target_info *tgt_info)
> {
> bool prog_extension = prog->type == BPF_PROG_TYPE_EXT;
> + bool prog_tracing = prog->type == BPF_PROG_TYPE_TRACING;
> const char prefix[] = "btf_trace_";
> int ret = 0, subprog = -1, i;
> const struct btf_type *t;
> @@ -20147,10 +20148,21 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
> bpf_log(log, "Can attach to only JITed progs\n");
> return -EINVAL;
> }
> - if (tgt_prog->type == prog->type) {
> - /* Cannot fentry/fexit another fentry/fexit program.
> - * Cannot attach program extension to another extension.
> - * It's ok to attach fentry/fexit to extension program.
> + if (prog_tracing) {
> + if (aux->attach_tracing_prog) {
> + /*
> + * Target program is an fentry/fexit which is already attached
> + * to another tracing program. More levels of nesting
> + * attachment are not allowed.
> + */
> + bpf_log(log, "Cannot nest tracing program attach more than once\n");
> + return -EINVAL;
> + }
> + } else if (tgt_prog->type == prog->type) {
> + /*
> + * To avoid potential call chain cycles, prevent attaching of a
> + * program extension to another extension. It's ok to attach
> + * fentry/fexit to extension program.
> */
> bpf_log(log, "Cannot recursively attach\n");
> return -EINVAL;
> @@ -20163,16 +20175,15 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
> * except fentry/fexit. The reason is the following.
> * The fentry/fexit programs are used for performance
> * analysis, stats and can be attached to any program
> - * type except themselves. When extension program is
> - * replacing XDP function it is necessary to allow
> - * performance analysis of all functions. Both original
> - * XDP program and its program extension. Hence
> - * attaching fentry/fexit to BPF_PROG_TYPE_EXT is
> - * allowed. If extending of fentry/fexit was allowed it
> - * would be possible to create long call chain
> - * fentry->extension->fentry->extension beyond
> - * reasonable stack size. Hence extending fentry is not
> - * allowed.
> + * type. When extension program is replacing XDP function
> + * it is necessary to allow performance analysis of all
> + * functions. Both original XDP program and its program
> + * extension. Hence attaching fentry/fexit to
> + * BPF_PROG_TYPE_EXT is allowed. If extending of
> + * fentry/fexit was allowed it would be possible to create
> + * long call chain fentry->extension->fentry->extension
> + * beyond reasonable stack size. Hence extending fentry
> + * is not allowed.
> */
> bpf_log(log, "Cannot extend fentry/fexit\n");
> return -EINVAL;
> --
> 2.41.0
>
