On Mon, 2022-04-04 at 16:41 -0700, Andrii Nakryiko wrote:
> Add BPF-side implementation of libbpf-provided USDT support. This
> consists of single header library, usdt.bpf.h, which is meant to be
> used
> from user's BPF-side source code. This header is added to the list of
> installed libbpf header, along bpf_helpers.h and others.
>
> BPF-side implementation consists of two BPF maps:
> - spec map, which contains "a USDT spec" which encodes information
> necessary to be able to fetch USDT arguments and other
> information
> (argument count, user-provided cookie value, etc) at runtime;
> - IP-to-spec-ID map, which is only used on kernels that don't
> support
> BPF cookie feature. It allows to lookup spec ID based on the
> place
> in user application that triggers USDT program.
>
> These maps have default sizes, 256 and 1024, which are chosen
> conservatively to not waste a lot of space, but handling a lot of
> common
> cases. But there could be cases when user application needs to either
> trace a lot of different USDTs, or USDTs are heavily inlined and
> their
> arguments are located in a lot of differing locations. For such cases
> it
> might be necessary to size those maps up, which libbpf allows to do
> by
> overriding BPF_USDT_MAX_SPEC_CNT and BPF_USDT_MAX_IP_CNT macros.
>
> It is an important aspect to keep in mind. Single USDT (user-space
> equivalent of kernel tracepoint) can have multiple USDT "call sites".
> That is, single logical USDT is triggered from multiple places in
> user
> application. This can happen due to function inlining. Each such
> inlined
> instance of USDT invocation can have its own unique USDT argument
> specification (instructions about the location of the value of each
> of
> USDT arguments). So while USDT looks very similar to usual uprobe or
> kernel tracepoint, under the hood it's actually a collection of
> uprobes,
> each potentially needing different spec to know how to fetch
> arguments.
>
> User-visible API consists of three helper functions:
> - bpf_usdt_arg_cnt(), which returns number of arguments of current
> USDT;
> - bpf_usdt_arg(), which reads value of specified USDT argument (by
> it's zero-indexed position) and returns it as 64-bit value;
> - bpf_usdt_cookie(), which functions like BPF cookie for USDT
> programs; this is necessary as libbpf doesn't allow specifying
> actual
> BPF cookie and utilizes it internally for USDT support
> implementation.
>
> Each bpf_usdt_xxx() APIs expect struct pt_regs * context, passed into
> BPF program. On kernels that don't support BPF cookie it is used to
> fetch absolute IP address of the underlying uprobe.
>
> usdt.bpf.h also provides BPF_USDT() macro, which functions like
> BPF_PROG() and BPF_KPROBE() and allows much more user-friendly way to
> get access to USDT arguments, if USDT definition is static and known
> to
> the user. It is expected that majority of use cases won't have to use
> bpf_usdt_arg_cnt() and bpf_usdt_arg() directly and BPF_USDT() will
> cover
> all their needs.
>
> Last, usdt.bpf.h is utilizing BPF CO-RE for one single purpose: to
> detect kernel support for BPF cookie. If BPF CO-RE dependency is
> undesirable, user application can redefine BPF_USDT_HAS_BPF_COOKIE to
> either a boolean constant (or equivalently zero and non-zero), or
> even
> point it to its own .rodata variable that can be specified from
> user's
> application user-space code. It is important that
> BPF_USDT_HAS_BPF_COOKIE is known to BPF verifier as static value
> (thus
> .rodata and not just .data), as otherwise BPF code will still contain
> bpf_get_attach_cookie() BPF helper call and will fail validation at
> runtime, if not dead-code eliminated.
>
> Reviewed-by: Alan Maguire <alan.maguire@xxxxxxxxxx>
> Signed-off-by: Andrii Nakryiko <andrii@xxxxxxxxxx>
> ---
> tools/lib/bpf/Makefile | 2 +-
> tools/lib/bpf/usdt.bpf.h | 256
> +++++++++++++++++++++++++++++++++++++++
> 2 files changed, 257 insertions(+), 1 deletion(-)
> create mode 100644 tools/lib/bpf/usdt.bpf.h
[...]
> diff --git a/tools/lib/bpf/usdt.bpf.h b/tools/lib/bpf/usdt.bpf.h
> new file mode 100644
> index 000000000000..60237acf6b02
> --- /dev/null
> +++ b/tools/lib/bpf/usdt.bpf.h
> @@ -0,0 +1,256 @@
[...]
> +/* Fetch USDT argument #*arg_num* (zero-indexed) and put its value
> into *res.
> + * Returns 0 on success; negative error, otherwise.
> + * On error *res is guaranteed to be set to zero.
> + */
> +static inline __noinline
> +int bpf_usdt_arg(struct pt_regs *ctx, __u64 arg_num, long *res)
> +{
> + struct __bpf_usdt_spec *spec;
> + struct __bpf_usdt_arg_spec *arg_spec;
> + unsigned long val;
> + int err, spec_id;
> +
> + *res = 0;
> +
> + spec_id = __bpf_usdt_spec_id(ctx);
> + if (spec_id < 0)
> + return -ESRCH;
> +
> + spec = bpf_map_lookup_elem(&__bpf_usdt_specs, &spec_id);
> + if (!spec)
> + return -ESRCH;
> +
> + if (arg_num >= BPF_USDT_MAX_ARG_CNT || arg_num >= spec-
> >arg_cnt)
> + return -ENOENT;
> +
> + arg_spec = &spec->args[arg_num];
> + switch (arg_spec->arg_type) {
> + case BPF_USDT_ARG_CONST:
> + /* Arg is just a constant ("-4@$-9" in USDT arg
> spec).
> + * value is recorded in arg_spec->val_off directly.
> + */
> + val = arg_spec->val_off;
> + break;
> + case BPF_USDT_ARG_REG:
> + /* Arg is in a register (e.g, "8@%rax" in USDT arg
> spec),
> + * so we read the contents of that register directly
> from
> + * struct pt_regs. To keep things simple user-space
> parts
> + * record offsetof(struct pt_regs, <regname>) in
> arg_spec->reg_off.
> + */
> + err = bpf_probe_read_kernel(&val, sizeof(val), (void
> *)ctx + arg_spec->reg_off);
> + if (err)
> + return err;
> + break;
> + case BPF_USDT_ARG_REG_DEREF:
> + /* Arg is in memory addressed by register, plus some
> offset
> + * (e.g., "-4@-1204(%rbp)" in USDT arg spec).
> Register is
> + * identified lik with BPF_USDT_ARG_REG case, and the
> offset
> + * is in arg_spec->val_off. We first fetch register
> contents
> + * from pt_regs, then do another user-space probe
> read to
> + * fetch argument value itself.
> + */
> + err = bpf_probe_read_kernel(&val, sizeof(val), (void
> *)ctx + arg_spec->reg_off);
> + if (err)
> + return err;
> + err = bpf_probe_read_user(&val, sizeof(val), (void
> *)val + arg_spec->val_off);
Is there a reason we always read 8 bytes here?
What if the user is interested in the last byte of a page?
[...]
