On Fri, May 31, 2024 at 04:36:48AM -0700, Tony Ambardar wrote: > Hi Jiri, > > On Fri, May 31, 2024 at 10:13:21AM +0200, Jiri Olsa wrote: > > On Fri, May 31, 2024 at 10:17:53AM +0800, Hengqi Chen wrote: > > > Hi Tony, > > > > > > On Fri, May 31, 2024 at 9:30 AM Tony Ambardar <tony.ambardar@xxxxxxxxx> wrote: > > > > > > > > Hello, > > > > > > > > For some time now I'm seeing multiple issues during BTF generation while > > > > building recent kernels targeting mips64el, and would appreciate some help > > > > to understand and fix the problems. > > > > > > > > Some relate to resolve_btfids: > > > > > > > > > LD vmlinux > > > > > BTFIDS vmlinux > > > > > WARN: resolve_btfids: unresolved symbol bpf_verify_pkcs7_signature > > > > > WARN: resolve_btfids: unresolved symbol bpf_session_cookie > > > > > WARN: resolve_btfids: unresolved symbol bpf_lookup_user_key > > > > > WARN: resolve_btfids: unresolved symbol bpf_lookup_system_key > > > > > WARN: resolve_btfids: unresolved symbol bpf_key_put > > > > > WARN: resolve_btfids: unresolved symbol bpf_iter_task_next > > > > > WARN: resolve_btfids: unresolved symbol bpf_iter_css_task_new > > > > > WARN: resolve_btfids: unresolved symbol bpf_get_file_xattr > > > > > WARN: resolve_btfids: unresolved symbol bpf_ct_insert_entry > > > > > WARN: resolve_btfids: unresolved symbol bpf_cgroup_release > > > > > WARN: resolve_btfids: unresolved symbol bpf_cgroup_from_id > > > > > WARN: resolve_btfids: unresolved symbol bpf_cgroup_acquire > > > > > WARN: resolve_btfids: unresolved symbol bpf_arena_free_pages > > > > > NM System.map > > > > > SORTTAB vmlinux > > > > > OBJCOPY vmlinux.32 > > > > > > > > These do not appear to be #ifdef-related and have similar past reports [1]. > > > > I can reproduce the warning just for bpf_session_cookie, > > which has fix in progress: > > https://lore.kernel.org/bpf/20240531071557.MvfIqkn7@xxxxxxxxxxxxx/T/#t > > I gather there are different root causes for these warnings. From the link, > the issue with bpf_session_cookie seems related to conditional compilation, > which have come up before on the mailing list IIRC. > > In comparison, consider my above warning for bpf_key_put, which is defined > in kernel/trace/bpf_trace.c. This kfunc is guarded by CONFIG_KEY, which is > enabled in my config and so not the issue. I can in fact see the global > text symbol for bpf_key_put in bpf_trace.o, but not in vmlinux. yes, that would be a problem, the resolve_btfids goes through the BTF lists and needs to resolve the function in vmlinux, when it's not there, it will output the warning above > > So I suspect the warnings might come from linker or optimization problems, > or perhaps even an issue related to the __bpf_kfunc annotation. WDYT? I can reproduce now resolve_btfids warnings: LD vmlinux BTFIDS vmlinux WARN: resolve_btfids: unresolved symbol bpf_verify_pkcs7_signature WARN: resolve_btfids: unresolved symbol bpf_session_is_return WARN: resolve_btfids: unresolved symbol bpf_session_cookie WARN: resolve_btfids: unresolved symbol bpf_lookup_user_key WARN: resolve_btfids: unresolved symbol bpf_lookup_system_key WARN: resolve_btfids: unresolved symbol bpf_key_put WARN: resolve_btfids: unresolved symbol bpf_iter_task_next WARN: resolve_btfids: unresolved symbol bpf_iter_css_task_new WARN: resolve_btfids: unresolved symbol bpf_get_file_xattr WARN: resolve_btfids: unresolved symbol bpf_ct_insert_entry WARN: resolve_btfids: unresolved symbol bpf_cgroup_release WARN: resolve_btfids: unresolved symbol bpf_cgroup_from_id WARN: resolve_btfids: unresolved symbol bpf_cgroup_acquire WARN: resolve_btfids: unresolved symbol bpf_arena_free_pages when I call bpf_key_put to make it 'used' it stays in vmlinux, so I wonder the __bpf_kfunc flags do not work properly on this cross compile chain.. and linker just optimize it away? #define __bpf_kfunc __used noinline thanks, jirka > > > > > > > > > > > I also see many pahole failures during BTF encoding of modules, such as: > > > > > > > > > CC [M] net/ipv6/netfilter/nft_fib_ipv6.mod.o > > > > > CC [M] net/ipv6/netfilter/ip6t_REJECT.mod.o > > > > > CC [M] net/psample/psample.mod.o > > > > > LD [M] crypto/cmac.ko > > > > > BTF [M] crypto/cmac.ko > > > > > die__process: DW_TAG_compile_unit, DW_TAG_type_unit, DW_TAG_partial_unit > > > > > or DW_TAG_skeleton_unit expected got member (0xd)! > > > > > > The issue seems to be related to elfutils. Have you tried build from > > > the latest elfutils source ? > > > I saw the latest MIPS backend in elfutils already implemented the > > > reloc_simple_type hook. > > > > hi, > > +1, could you also check the pahole version you used? > > No luck I'm afraid with using the latest elfutils as suggested. > > I used pahole v1.26 in my original testing for this bug report, as noted > below (might have been buried a bit). > > > > > jirka > > > > SNIP > > > > > > Details of the git commit and build environment are as follows: > > > > > > > > > $ git log -1 --oneline bpf/master > > > > > 9dfdb706e164 (bpf/master) selftests/bpf: fix inet_csk_accept prototype in > > > > > test_sk_storage_tracing.c > > > > > > > > > > $ lsb_release -a > > > > > Description: Ubuntu 22.04.4 LTS > > > > > > > > > > $ cat gcc-compile.txt > > > > > ARCH=mips CROSS_COMPILE=mips64el-linux-gnuabi64- CC="ccache ${CROSS_COMPILE}gcc" make -j6 > > > > > > > > > > $ mips64el-linux-gnuabi64-gcc --version > > > > > mips64el-linux-gnuabi64-gcc (Ubuntu 10.3.0-1ubuntu1) 10.3.0 > > > > > > > > > > $ mips64el-linux-gnuabi64-ld --version > > > > > GNU ld (GNU Binutils for Ubuntu) 2.38 > > > > > > > > > > $ pahole --version > > > > > v1.26 > > > > SNIP > > > > > > I'd be grateful if some of the BTF/pahole experts could please review this > > > > issue and share next steps or other details I might provide. > > > > > > > > Thanks, > > > > Tony Ambardar > > > > > > > > Link: https://lore.kernel.org/all/202401211357.OCX9yllM-lkp@xxxxxxxxx/ [1] > > > > Link: https://github.com/acmel/dwarves/issues/45 [2] > > > > > > Cheers, > > > Hengqi > > Thanks, > Tony
