Re: [RESEND] BTF is not generated for gcc-built kernel with the latest pahole

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 26/07/2023 12:07, Masami Hiramatsu (Google) wrote:
> Hi Jiri,
> 
> On Wed, 26 Jul 2023 12:04:08 +0200
> Jiri Olsa <olsajiri@xxxxxxxxx> wrote:
> 
>> On Wed, Jul 26, 2023 at 10:25:34AM +0900, Masami Hiramatsu wrote:
>>> Hello,
>>> (I resend this because kconfig was too big)
>>>
>>> I found that BTF is not generated for gcc-built kernel with that latest
>>> pahole (v1.25).
>>
>> hi,
>> I can't reproduce on my setup with your .config
>>
>> does 'bpftool btf dump file ./vmlinux' show any error?
>>
>> is there any error in the kernel build output?
> 
> Yes, here it is. I saw these 2 lines.
> 
> die__process: DW_TAG_compile_unit, DW_TAG_type_unit, DW_TAG_partial_unit or DW_TAG_skeleton_unit expected got INVALID (0x0)!
> die__process: DW_TAG_compile_unit, DW_TAG_type_unit, DW_TAG_partial_unit or DW_TAG_skeleton_unit expected got INVALID (0x0)!

This is strange, looks like some CUs were encoded incorrectly or we are
parsing incorrectly. The error originates in die__process() and happens
if the dwarf_tag() associated with the DIE isn't an expected unit; it's
not even a valid tag value (0) it looks like. I've not built with gcc 13
yet so it's possible that's the reason you're seeing this, I'll try to
reproduce it..

> 
>>
>>> When I'm using the distro origin pahole (v1.22) it works. (I also checked
>>> v1.23 and v1.24, both partially generated BTF)
>>>
>>> e.g.
>>>
>>> # echo 'f kfree $arg*' >> /sys/kernel/tracing/dynamic_events
>>> sh: write error: Invalid argument
>>>
>>> # cat /sys/kernel/tracing/error_log 
>>> [   21.595724] trace_fprobe: error: BTF is not available or not supported
>>>   Command: f kfree $arg*
>>>                    ^
>>> [   21.596032] trace_fprobe: error: Invalid $-valiable specified
>>>   Command: f kfree $arg*
>>>                    ^
>>>
>>> / # strings /sys/kernel/btf/vmlinux | grep kfree
>>
>> hm, if you have this file present, you have BTF in
> 
> Yes, it seems that the BTF itself is generated, but many entries seems
> dropped compared with pahole v1.22. So, if a given symbol has BTF, (e.g. 
> kfree_rcu_batch_init) it works.
>

Yep, BPF generation is more selective about what it emits in 1.25 to
avoid cases where a kernel function signature is ambiguous (multiple
functions of the same name with different signatures) or where it has
unexpected register use. You can observe this via pahole's --verbose
option (a lot of outut is emitted):

In a built kernel directory (where unstripped vmlinux is present):
$ PAHOLE_FLAGS=$(./scripts/pahole_flags)
$ PAHOLE=/usr/local/bin/pahole
$ pahole --verbose -J $PAHOLE_FLAGS vmlinux > /tmp/pahole.out

If you want to investigate why a function has been left out, look for
"skipping" verbose output like this:

skipping addition of 'access_error'(access_error) due to multiple
inconsistent function prototypes
skipping addition of
'acpi_ex_convert_to_object_type_string'(acpi_ex_convert_to_object_type_string.isra.0)
due to unexpected register used for parameter

FWIW I see most of the below in my BTF output on bpf-next, though I am
missing a few, possibly due to differing config options since they don't
appear in kallsyms either. I don't see the DWARF tag label not handled
messages so it's possible that's a symptom of something

I suspect however some form of corruption in the DWARF representation
may be the reason a lot of these are missing. Would be worth trying
to "objdump -g vmlinux >vmlinux.dwarf" (file will be huge tho) I suspect.


>>
>>> kfree_on_online
>>> maybe_kfree_parameter
>>> trace_event_raw_rcu_invoke_kfree_bulk_callback
>>> trace_event_data_offsets_rcu_invoke_kfree_bulk_callback
>>> btf_trace_rcu_invoke_kfree_bulk_callback
>>> early_boot_kfree_rcu
>>> __bpf_trace_rcu_invoke_kfree_bulk_callback
>>> perf_trace_rcu_invoke_kfree_bulk_callback
>>> trace_event_raw_event_rcu_invoke_kfree_bulk_callback
>>> trace_raw_output_rcu_invoke_kfree_bulk_callback
>>> __probestub_rcu_invoke_kfree_bulk_callback
>>> __traceiter_rcu_invoke_kfree_bulk_callback
>>> kfree_rcu_cpu_work
>>> kfree_rcu_cpu
>>> kfree_rcu_batch_init
>>> kfree_rcu_scheduler_running
>>> kfree_rcu_shrink_scan
>>> kfree_rcu_shrink_count
>>> kfree_rcu_monitor
>>> kfree_rcu_work
>>>
>>>
>>> Here is the gcc version which I'm using.
>>>
>>> gcc version 11.3.0 (Ubuntu 11.3.0-1ubuntu1~22.04.1)
>>
>> I tried with gcc (GCC) 13.1.1 20230614 (Red Hat 13.1.1-4)
>> and latest pahole (master branch)
> 
> Curiously, with Clang 16.0.0, it works (but many different errors are shown,
> see below *).
> So the combination of gcc/clang and pahole version can affect it.
> 
>>
>>>
>>> I also attached the kernel config file.
>>>
>>> What is the recommended combination of the tools?
>>> Should I use Clang to build the kernel for BTF?
>>
>> should work fine with both gcc and clang
> 
> And I guess it depends on compiler version, doesn't it?
> 
> Thank you,
> 
> 
> (*)
>   BTF     .btf.vmlinux.bin.o
> die__process_unit: DW_TAG_label (0xa) @ <0x4b138> not handled!
> die__process_unit: tag not supported 0xa (label)!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b241> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b263> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b290> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b2c0> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b2eb> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b317> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b33a> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b35a> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b37d> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b39e> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b3c4> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b3e7> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b40f> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b435> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b457> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b477> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b4a5> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b4d5> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b505> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b530> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b560> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b585> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b5b2> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b5d9> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b605> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b62e> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b652> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b670> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b694> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b6b3> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b6d9> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b705> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b72b> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b753> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b77f> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b7b3> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b7e4> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b811> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b83c> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b869> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b88c> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b8bd> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b8e7> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b90b> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b930> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b960> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b997> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4b9ce> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4ba00> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4ba24> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4ba4d> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4ba89> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4babc> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bae4> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bb0b> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bb2e> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bb4e> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bb6d> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bb8a> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bba8> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bbc5> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bbe1> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bc01> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bc1c> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bc38> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bc58> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bc79> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bc95> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4bcb2> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x7efd6> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x7effe> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x7f11c> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x7f143> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x7f178> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x7f1a4> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x7f1ca> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x7f1fb> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x7f22f> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x7f25a> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x7f28d> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x7f2b7> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x1eea81> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x1eea9d> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x1eeac3> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x1eeaf3> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x1eeb0f> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x1eeb30> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x1eeb59> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x340734> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4c6b4a> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4c6b67> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4c6b8a> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4c6ba5> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4c6bc4> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4c6bea> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4c6c07> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4c6c2a> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4c6c4e> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4c6c79> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4c6c9b> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4c6cc3> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4c6ceb> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x4c6d15> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x30626ac> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x30626cd> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x3062814> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x3062833> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x3062b8b> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f623f6> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f62416> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f62437> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f62458> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f6280b> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f62b09> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f62b37> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f62c45> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f62c60> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f62d69> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f62e8d> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f6305a> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f63ec2> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f63edf> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f63efc> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f63f19> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f63f36> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f63f5b> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f63f80> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f63fa5> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f63fca> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f6fcae> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f6fcc7> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f6fed7> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f6fef0> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f6fdcb> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70352> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f7036f> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70394> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f703b1> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f703d6> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f703f3> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70418> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70435> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f7045a> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f7057b> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f705a6> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f705cf> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f705f8> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70621> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f7064a> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70673> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f7069c> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f706c5> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f706ee> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70716> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f7073e> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70767> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70790> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f707b9> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f707e2> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f7080b> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70834> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70864> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70892> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f708c0> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f708ee> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f7091c> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f7094a> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70978> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f709a6> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f709d4> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70a01> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70a2e> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70a5c> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70a8a> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70ab8> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70ae6> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70b14> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70b42> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70b72> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70ba0> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70bce> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70bfc> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70c2a> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70c58> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70c86> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70cb4> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70ce2> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70d0f> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70d3c> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70d6a> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70d98> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70dc6> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70df4> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70e22> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70e50> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70e71> not handled!
> die__process_unit: DW_TAG_label (0xa) @ <0x5f70e94> not handled!
>   LD      .tmp_vmlinux.kallsyms1
> 
>>
>> jirka
>>
>>>
>>> Thank you,
>>>
>>> -- 
>>> Masami Hiramatsu (Google) <mhiramat@xxxxxxxxxx>
>>
>>
> 
> 




[Index of Archives]     [Linux Samsung SoC]     [Linux Rockchip SoC]     [Linux Actions SoC]     [Linux for Synopsys ARC Processors]     [Linux NFS]     [Linux NILFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]


  Powered by Linux