Hi, On Tue, 12 Apr 2022 15:51:43 -0700 Andrii Nakryiko <andrii.nakryiko@xxxxxxxxx> wrote: > On Mon, Apr 11, 2022 at 5:49 PM Masami Hiramatsu <mhiramat@xxxxxxxxxx> wrote: > > > > On Mon, 11 Apr 2022 15:15:40 -0700 > > Andrii Nakryiko <andrii.nakryiko@xxxxxxxxx> wrote: > > > > > > +#define DEBUGFS "/sys/kernel/debug/tracing/" > > > > + > > > > +static int get_syms(char ***symsp, size_t *cntp) > > > > +{ > > > > + size_t cap = 0, cnt = 0, i; > > > > + char *name, **syms = NULL; > > > > + struct hashmap *map; > > > > + char buf[256]; > > > > + FILE *f; > > > > + int err; > > > > + > > > > + /* > > > > + * The available_filter_functions contains many duplicates, > > > > + * but other than that all symbols are usable in kprobe multi > > > > + * interface. > > > > + * Filtering out duplicates by using hashmap__add, which won't > > > > + * add existing entry. > > > > + */ > > > > + f = fopen(DEBUGFS "available_filter_functions", "r"); > > > > > > I'm really curious how did you manage to attach to everything in > > > available_filter_functions because when I'm trying to do that I fail. > > > available_filter_functions has a bunch of functions that should not be > > > attachable (e.g., notrace functions). Look just at __bpf_tramp_exit: > > > > > > void notrace __bpf_tramp_exit(struct bpf_tramp_image *tr); > > > > Hmm, this sounds like a bug in ftrace side. IIUC, the > > "available_filter_functions" only shows the functions which is NOT > > instrumented by mcount, we should not see any notrace functions on it. > > > > Technically, this is done by __no_instrument_function__ attribute. > > > > #if defined(CC_USING_HOTPATCH) > > #define notrace __attribute__((hotpatch(0, 0))) > > #elif defined(CC_USING_PATCHABLE_FUNCTION_ENTRY) > > #define notrace __attribute__((patchable_function_entry(0, 0))) > > #else > > #define notrace __attribute__((__no_instrument_function__)) > > #endif > > > > > > > > So first, curious what I am doing wrong or rather why it succeeds in > > > your case ;) > > > > > > But second, just wanted to plea to "fix" available_filter_functions to > > > not list stuff that should not be attachable. Can you please take a > > > look and checks what's going on there and why do we have notrace > > > functions (and what else should *NOT* be there)? > > > > Can you share how did you reproduce the issue? I'll check it. > > > > $ sudo cat /sys/kernel/debug/tracing/available_filter_functions | grep > __bpf_tramp > __bpf_tramp_image_release > __bpf_tramp_image_put_rcu_tasks > __bpf_tramp_image_put_rcu > __bpf_tramp_image_put_deferred > __bpf_tramp_exit > > > __bpf_tramp_exit is notrace function, so shouldn't be here. Notice > that __bpf_tramp_enter (which is also notrace) are not in > available_filter_functions. OK, I also confirmed that __bpf_tramp_exit is listed. (others seems no notrace) /sys/kernel/tracing # cat available_filter_functions | grep __bpf_tramp __bpf_tramp_image_release __bpf_tramp_image_put_rcu __bpf_tramp_image_put_rcu_tasks __bpf_tramp_image_put_deferred __bpf_tramp_exit My gcc is older one. gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1) But it seems that __bpf_tramp_exit() doesn't call __fentry__. (I objdump'ed) ffffffff81208270 <__bpf_tramp_exit>: ffffffff81208270: 55 push %rbp ffffffff81208271: 48 89 e5 mov %rsp,%rbp ffffffff81208274: 53 push %rbx ffffffff81208275: 48 89 fb mov %rdi,%rbx ffffffff81208278: e8 83 70 ef ff callq ffffffff810ff300 <__rcu_read_lock> ffffffff8120827d: 31 d2 xor %edx,%edx > > So it's quite bizarre and inconsistent. Indeed. I guess there is a bug in scripts/recordmcount.pl. Thank you, > > > Thank you, > > > > > > -- > > Masami Hiramatsu <mhiramat@xxxxxxxxxx> -- Masami Hiramatsu <mhiramat@xxxxxxxxxx>