On Mon, Jun 03, 2024 at 08:47:24PM -0700, Tony Ambardar wrote: > Hi Mark, > > On Mon, Jun 03, 2024 at 09:18:33PM +0200, Mark Wielaard wrote: > > On Mon, Jun 03, 2024 at 02:40:45PM -0300, Arnaldo Carvalho de Melo wrote: > > > Couldn't find a way to ask eu-readelf for more verbose output, where we > > > could perhaps get some clue as to why it produces nothing while binutils > > > readelf manages to grok it, Mark, do you know some other way to ask > > > eu-readelf to produce more debug output? > > > > > > I'm unsure if the netdevsim.ko file was left in a semi encoded BTF state > > > that then made eu-readelf to not be able to process it while pahole, > > > that uses eltuils' libraries, was able to process the first two CUs for > > > a kernel module and all the CUs for the vmlinux file :-\ > > > > > > Mark, the whole thread is available at: > > > > > > https://lore.kernel.org/all/Zl3Zp5r9m6X_i_J4@x1/T/#u > > > > I haven't looked at the vmlinux file. But for the .ko file the issue > > is that the elfutils MIPS backend isn't complete. Specifically MIPS > > relocations aren't recognized (and so cannot be applied). There are > > some pending patches which try to fix that: > > > > https://patchwork.sourceware.org/project/elfutils/list/?series=31601 > > Earlier in the thread, Hengqi Chen pointed out the latest elfutils backend > work for MIPS, and I locally rebuilt elfutils and then pahole from their > respective next/main branches. For elfutils, main (935ee131cf7c) includes > > e259f126 Support Mips architecture > f2acb069 stack: Fix stack unwind failure on mips > db33cb0c backends: Add register_info, return_value_location, core_note mips > > which partially applies the patchwork series but leaves out the support for > readelf, strip, and elflint. > > I believe this means the vmlinux and .ko files I shared are OK, or is there > more backend work needed for MIPS? > > The bits missing in eu-readelf would explain the blank output both Arnaldo > and I see from "$ eu-readelf -winfo vmlinux". I tried rebuilding with the > patchwork readelf patch locally but ran into merge conflicts. > > CCing Ying Huang for any more insight. > > Thanks, > Tony Hello all, A short update, starting with answering my own question. No, apparently the above commits *do not* complete the backend work. Ying Huang submitted additional related patches since March 5: [1][2] strip: Adapt src/strip -o -f on mips readelf: Adapt src/readelf -h/-S/-r/-w/-l/-d/-a on mips elflint: adapt src/elflint --gnu src/nm on mips test: Add mips in run-allregs.sh and run-readelf-mixed-corenote.sh Despite the titles, these patches do include core backend changes for MIPS. I resolved the various merge conflicts [3], rebuilt elfutils, and retested kernel builds to now find: - pahole is able to read DWARF[45] info and create .BTF for modules - resolve_btfids can successfully patch .BTF_ids in modules - kernel successfully loads modules with BTF and kfuncs (tested 6.6 LTS) Huzzah! Ying: Thank you for developing these MIPS patches. In your view, are the MIPS changes now complete, or do you plan further updates that might improve or impact parsing DWARF debug/reloc info in apps like pahole? Mark: Given that BTF usage on Linux/MIPS is basically broken without these patches, could I request some of your review time for them to be merged? If it's helpful, my branch [3] includes all patches with conflicts fixed, and I also successfully ran the elfutils self-tests (including MIPS from Ying). Please feel free to add for these patches: Tested-by: Tony Ambardar <Tony.Ambardar@xxxxxxxxx> Arnaldo: Your stepping through DWARF/reloc diagnostics earlier was helpful. Thanks! I reran your tests with the updated elfutils and latest pahole (pre-1.27), and then found: - everything that worked before, still works - your observations from "btfdiff vmlinux" and 'struct dma_chan' persist - we now see expected output from "eu-readelf -winfo netdevsim.ko" Regarding pahole, DWARF parsing and BTF generation now works: (with no more die__process: error messages seen) kodidev:~/linux$ pahole -F dwarf netdevsim.ko |wc -l 14504 but strangely pahole still doesn't read its own generated BTF: kodidev:~/linux$ pahole -F btf netdevsim.ko libbpf: Invalid BTF string section pahole: file 'netdevsim.ko' has no btf type information. Poking inside a little further: kodidev:~/linux$ ltrace -S pahole -F btf netdevsim.ko [...] argp_parse(0x563d47da42a0, 4, 0x7ffd5e552698, 0 <unfinished ...> SYS_318(0x7fc385bf84d8, 8, 1, 0x7fc385ce9908) = 8 SYS_brk(0) = 0x563d47e37000 SYS_brk(0x563d47e58000) = 0x563d47e58000 <... argp_parse resumed> ) = 0 dwarves__init(0x20000, 0, -4096, 213) = 0 dwarves__resolve_cacheline_size(0x563d47da40c0, 0, 24, 213) = 64 cus__new(5, 0, 64, 213) = 0x563d47e372a0 memset(0x563d47da44c0, ' ', 127) = 0x563d47da44c0 strlen("/sys/kernel/btf/") = 16 strncmp("netdevsim.ko", "/sys/kernel/btf/", 16) = 63 cus__load_files(0x563d47e372a0, 0x563d47da40c0, 0x7ffd5e5526b0, 0x563d47da40c0 <unfinished ...> SYS_openat(0xffffff9c, 0x7ffd5e554603, 0x80000, 0) = 3 SYS_newfstatat(3, 0x7fc385baf44f, 0x7ffd5e552210, 4096) = 0 SYS_read(3, "\177ELF\002\001\001", 4096) = 4096 SYS_close(3) = 0 SYS_openat(0xffffff9c, 0x7ffd5e554603, 0x80000, 0) = 3 SYS_fcntl(3, 1, 0, 0x7fc3859de2f0) = 1 SYS_newfstatat(3, 0x7fc385baf44f, 0x7ffd5e5521f0, 4096) = 0 SYS_pread(3, 0x7ffd5e5521f0, 64, 0) = 64 SYS_pread(3, 0x563d47e3e470, 3520, 0x4f60c8) = 3520 SYS_pread(3, 0x563d47e3f240, 525, 0x4f5eb8) = 525 SYS_pread(3, 0x563d47e3f460, 0x45dd, 0x2b1fb6) = 0x45dd SYS_write(2, "libbpf: Invalid BTF string secti"..., 35libbpf: Invalid BTF string section ) = 35 SYS_close(3) = 0 <... cus__load_files resumed> ) = 0xffffffff access("netdevsim.ko", 4 <unfinished ...> SYS_access("netdevsim.ko", 04) = 0 <... access resumed> ) = 0 fprintf(0x7fc385bf26a0, "pahole: file '%s' has no %s type"..., "netdevsim.ko", "btf" <unfinished ...> SYS_write(2, "pahole: file 'netdevsim.ko' has "..., 57pahole: file 'netdevsim.ko' has no btf type information. ) = 57 <... fprintf resumed> ) = 57 SYS_exit_group(1 <no return ...> +++ exited (status 1) +++ Could you help investigate this further? Maybe a libbpf issue? For the record, I also tried building pahole with embedded libbpf 1.4.3 without any change. (side note: please make pahole --version also cover libbpf) Many thanks everyone for your help, Tony [1]: https://patchwork.sourceware.org/project/elfutils/list/?series=31601 [2]: https://patchwork.sourceware.org/project/elfutils/list/?series=34310 [3]: https://github.com/guidosarducci/elfutils/commits/main-fix-mips-support-reloc/
