On 26/07/2023 12:07, Masami Hiramatsu (Google) wrote: > Hi Jiri, > > On Wed, 26 Jul 2023 12:04:08 +0200 > Jiri Olsa <olsajiri@xxxxxxxxx> wrote: > >> On Wed, Jul 26, 2023 at 10:25:34AM +0900, Masami Hiramatsu wrote: >>> Hello, >>> (I resend this because kconfig was too big) >>> >>> I found that BTF is not generated for gcc-built kernel with that latest >>> pahole (v1.25). >> >> hi, >> I can't reproduce on my setup with your .config >> >> does 'bpftool btf dump file ./vmlinux' show any error? >> >> is there any error in the kernel build output? > > Yes, here it is. I saw these 2 lines. > > die__process: DW_TAG_compile_unit, DW_TAG_type_unit, DW_TAG_partial_unit or DW_TAG_skeleton_unit expected got INVALID (0x0)! > die__process: DW_TAG_compile_unit, DW_TAG_type_unit, DW_TAG_partial_unit or DW_TAG_skeleton_unit expected got INVALID (0x0)! This is strange, looks like some CUs were encoded incorrectly or we are parsing incorrectly. The error originates in die__process() and happens if the dwarf_tag() associated with the DIE isn't an expected unit; it's not even a valid tag value (0) it looks like. I've not built with gcc 13 yet so it's possible that's the reason you're seeing this, I'll try to reproduce it.. > >> >>> When I'm using the distro origin pahole (v1.22) it works. (I also checked >>> v1.23 and v1.24, both partially generated BTF) >>> >>> e.g. >>> >>> # echo 'f kfree $arg*' >> /sys/kernel/tracing/dynamic_events >>> sh: write error: Invalid argument >>> >>> # cat /sys/kernel/tracing/error_log >>> [ 21.595724] trace_fprobe: error: BTF is not available or not supported >>> Command: f kfree $arg* >>> ^ >>> [ 21.596032] trace_fprobe: error: Invalid $-valiable specified >>> Command: f kfree $arg* >>> ^ >>> >>> / # strings /sys/kernel/btf/vmlinux | grep kfree >> >> hm, if you have this file present, you have BTF in > > Yes, it seems that the BTF itself is generated, but many entries seems > dropped compared with pahole v1.22. So, if a given symbol has BTF, (e.g. > kfree_rcu_batch_init) it works. > Yep, BPF generation is more selective about what it emits in 1.25 to avoid cases where a kernel function signature is ambiguous (multiple functions of the same name with different signatures) or where it has unexpected register use. You can observe this via pahole's --verbose option (a lot of outut is emitted): In a built kernel directory (where unstripped vmlinux is present): $ PAHOLE_FLAGS=$(./scripts/pahole_flags) $ PAHOLE=/usr/local/bin/pahole $ pahole --verbose -J $PAHOLE_FLAGS vmlinux > /tmp/pahole.out If you want to investigate why a function has been left out, look for "skipping" verbose output like this: skipping addition of 'access_error'(access_error) due to multiple inconsistent function prototypes skipping addition of 'acpi_ex_convert_to_object_type_string'(acpi_ex_convert_to_object_type_string.isra.0) due to unexpected register used for parameter FWIW I see most of the below in my BTF output on bpf-next, though I am missing a few, possibly due to differing config options since they don't appear in kallsyms either. I don't see the DWARF tag label not handled messages so it's possible that's a symptom of something I suspect however some form of corruption in the DWARF representation may be the reason a lot of these are missing. Would be worth trying to "objdump -g vmlinux >vmlinux.dwarf" (file will be huge tho) I suspect. >> >>> kfree_on_online >>> maybe_kfree_parameter >>> trace_event_raw_rcu_invoke_kfree_bulk_callback >>> trace_event_data_offsets_rcu_invoke_kfree_bulk_callback >>> btf_trace_rcu_invoke_kfree_bulk_callback >>> early_boot_kfree_rcu >>> __bpf_trace_rcu_invoke_kfree_bulk_callback >>> perf_trace_rcu_invoke_kfree_bulk_callback >>> trace_event_raw_event_rcu_invoke_kfree_bulk_callback >>> trace_raw_output_rcu_invoke_kfree_bulk_callback >>> __probestub_rcu_invoke_kfree_bulk_callback >>> __traceiter_rcu_invoke_kfree_bulk_callback >>> kfree_rcu_cpu_work >>> kfree_rcu_cpu >>> kfree_rcu_batch_init >>> kfree_rcu_scheduler_running >>> kfree_rcu_shrink_scan >>> kfree_rcu_shrink_count >>> kfree_rcu_monitor >>> kfree_rcu_work >>> >>> >>> Here is the gcc version which I'm using. >>> >>> gcc version 11.3.0 (Ubuntu 11.3.0-1ubuntu1~22.04.1) >> >> I tried with gcc (GCC) 13.1.1 20230614 (Red Hat 13.1.1-4) >> and latest pahole (master branch) > > Curiously, with Clang 16.0.0, it works (but many different errors are shown, > see below *). > So the combination of gcc/clang and pahole version can affect it. > >> >>> >>> I also attached the kernel config file. >>> >>> What is the recommended combination of the tools? >>> Should I use Clang to build the kernel for BTF? >> >> should work fine with both gcc and clang > > And I guess it depends on compiler version, doesn't it? > > Thank you, > > > (*) > BTF .btf.vmlinux.bin.o > die__process_unit: DW_TAG_label (0xa) @ <0x4b138> not handled! > die__process_unit: tag not supported 0xa (label)! > die__process_unit: DW_TAG_label (0xa) @ <0x4b241> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b263> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b290> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b2c0> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b2eb> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b317> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b33a> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b35a> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b37d> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b39e> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b3c4> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b3e7> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b40f> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b435> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b457> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b477> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b4a5> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b4d5> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b505> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b530> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b560> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b585> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b5b2> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b5d9> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b605> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b62e> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b652> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b670> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b694> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b6b3> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b6d9> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b705> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b72b> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b753> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b77f> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b7b3> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b7e4> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b811> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b83c> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b869> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b88c> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b8bd> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b8e7> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b90b> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b930> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b960> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b997> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4b9ce> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4ba00> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4ba24> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4ba4d> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4ba89> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4babc> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bae4> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bb0b> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bb2e> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bb4e> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bb6d> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bb8a> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bba8> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bbc5> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bbe1> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bc01> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bc1c> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bc38> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bc58> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bc79> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bc95> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4bcb2> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x7efd6> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x7effe> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x7f11c> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x7f143> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x7f178> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x7f1a4> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x7f1ca> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x7f1fb> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x7f22f> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x7f25a> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x7f28d> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x7f2b7> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x1eea81> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x1eea9d> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x1eeac3> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x1eeaf3> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x1eeb0f> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x1eeb30> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x1eeb59> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x340734> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4c6b4a> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4c6b67> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4c6b8a> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4c6ba5> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4c6bc4> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4c6bea> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4c6c07> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4c6c2a> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4c6c4e> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4c6c79> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4c6c9b> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4c6cc3> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4c6ceb> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x4c6d15> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x30626ac> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x30626cd> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x3062814> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x3062833> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x3062b8b> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f623f6> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f62416> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f62437> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f62458> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f6280b> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f62b09> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f62b37> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f62c45> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f62c60> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f62d69> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f62e8d> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f6305a> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f63ec2> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f63edf> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f63efc> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f63f19> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f63f36> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f63f5b> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f63f80> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f63fa5> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f63fca> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f6fcae> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f6fcc7> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f6fed7> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f6fef0> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f6fdcb> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70352> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f7036f> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70394> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f703b1> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f703d6> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f703f3> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70418> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70435> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f7045a> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f7057b> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f705a6> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f705cf> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f705f8> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70621> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f7064a> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70673> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f7069c> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f706c5> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f706ee> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70716> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f7073e> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70767> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70790> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f707b9> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f707e2> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f7080b> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70834> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70864> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70892> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f708c0> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f708ee> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f7091c> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f7094a> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70978> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f709a6> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f709d4> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70a01> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70a2e> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70a5c> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70a8a> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70ab8> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70ae6> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70b14> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70b42> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70b72> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70ba0> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70bce> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70bfc> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70c2a> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70c58> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70c86> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70cb4> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70ce2> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70d0f> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70d3c> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70d6a> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70d98> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70dc6> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70df4> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70e22> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70e50> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70e71> not handled! > die__process_unit: DW_TAG_label (0xa) @ <0x5f70e94> not handled! > LD .tmp_vmlinux.kallsyms1 > >> >> jirka >> >>> >>> Thank you, >>> >>> -- >>> Masami Hiramatsu (Google) <mhiramat@xxxxxxxxxx> >> >> > >
