On Fri, Oct 9, 2020 at 9:22 AM Arnaldo Carvalho de Melo <acme@xxxxxxxxxx> wrote: > > Em Thu, Oct 08, 2020 at 04:39:52PM -0700, Andrii Nakryiko escreveu: > > This patch set switches pahole to use libbpf-provided BTF loading and encoding > > APIs. This reduces pahole's own BTF encoding code, speeds up the process, > > reduces amount of RAM needed for DWARF-to-BTF conversion. Also, pahole finally > > gets support to generating BTF for cross-compiled ELF binaries with different > > endianness (patch #8). > > > > Additionally, patch #3 fixes previously missed problem with invalid array > > index type generation. > > > > Patches #4-7 are speeding up DWARF-to-BTF convertion/dedup pretty > > significantly, saving overall about 9 seconds out of current 27 or so. > > > > Patch #5 revamps how per-CPU BTF variables are emitted, eliminating repeated > > and expensive looping over ELF symbols table. The critical detail that took > > few hours of investigation is that when DW_AT_variable has > > DW_AT_specification, variable address (to correlate with symbol's address) has > > to be taken before specification is followed. > > > > More details could be found in respective patches. > > > > v1->v2: > > - rebase on latest dwarves master and fix var->spec's address problem. > > Thanks, I applied all of them, tested and reproduced the performance > gains, great work! Great, thanks a lot, Arnaldo! Next step is adding BTF to kernel modules, where module's BTF will be an "extension" of vmlinux's BTF, with only a minimal set of new types used/added in the module, that are not available in vmlinux. This should make per-module BTF really tiny. > > I'll do some more testing on encoding a vmlinux for some big endian arch > on my x86_64 workstation and then push things publicly. > > If Hao find any issues we can fix in a follow up patch. > > I also added the people involved in the discussion about cross builds > failing, please take a look, I'm pushing now to a tmp.libbtf_encoder so > that you can test it from there, ok? > > - Arnaldo > > > Cc: Arnaldo Carvalho de Melo <acme@xxxxxxxxxx> > > > > Andrii Nakryiko (8): > > btf_loader: use libbpf to load BTF > > btf_encoder: use libbpf APIs to encode BTF type info > > btf_encoder: fix emitting __ARRAY_SIZE_TYPE__ as index range type > > btf_encoder: discard CUs after BTF encoding > > btf_encoder: revamp how per-CPU variables are encoded > > dwarf_loader: increase the size of lookup hash map > > strings: use BTF's string APIs for strings management > > btf_encoder: support cross-compiled ELF binaries with different > > endianness > > > > btf_encoder.c | 370 +++++++++++++++------------ > > btf_loader.c | 244 +++++++----------- > > ctf_encoder.c | 2 +- > > dwarf_loader.c | 2 +- > > libbtf.c | 661 +++++++++++++++++++++---------------------------- > > libbtf.h | 41 ++- > > libctf.c | 14 +- > > libctf.h | 4 +- > > pahole.c | 2 +- > > strings.c | 91 +++---- > > strings.h | 32 +-- > > 11 files changed, 645 insertions(+), 818 deletions(-) > > > > -- > > 2.24.1 > > > > -- > > - Arnaldo
