On Thu, Jun 3, 2021 at 7:57 AM Arnaldo Carvalho de Melo <acme@xxxxxxxxxx> wrote: > > Em Sat, May 29, 2021 at 05:40:17PM -0700, Andrii Nakryiko escreveu: > > On Fri, May 28, 2021 at 12:45 PM Arnaldo Carvalho de Melo <arnaldo.melo@xxxxxxxxx> wrote: > > > commit b579a18a1ea0ee84b90b5302f597dda2edf2f61b > > > Author: Arnaldo Carvalho de Melo <acme@xxxxxxxxxx> > > > Date: Fri May 28 16:41:30 2021 -0300 > > > > > > pahole: Allow encoding BTF into a detached file > > > > > > Previously the newly encoded BTF info was stored into a ELF section in > > > the file where the DWARF info was obtained, but it is useful to just > > > dump it into a separate file, do it. > > > > > > Requested-by: Andrii Nakryiko <andrii@xxxxxxxxxx> > > > Signed-off-by: Arnaldo Carvalho de Melo <acme@xxxxxxxxxx> > > > > > > > Looks good, see few minor comments below. At some point it probably > > would make sense to formalize "btf_encoder" as a struct with its own > > state instead of passing in multiple variables. It would probably also > > Take a look at the tmp.master branch at: > > https://git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=tmp.master Oh wow, that's a lot of commits! :) Great that you decided to do this refactoring, thanks! > > that btf_elf class isn't used anymore by btf_loader, that uses only > libbpf's APIs, and now we have a btf_encoder class with all the globals, > etc, more baby steps are needed to finally ditch btf_elf altogether and > move on to the parallelization. So do you plan to try to parallelize as a next step? I'm pretty confident about BTF encoding part: dump each CU into its own BTF, use btf__add_type() to merge multiple BTFs together. Just need to re-map IDs (libbpf internally has API to visit each field that contains type_id, it's well-defined enough to expose that as a public API, if necessary). Then final btf_dedup(). But the DWARF loading and parsing part is almost a black box to me, so I'm not sure how much work it would involve. > > I'm doing 'pahole -J vmlinux && btfdiff' after each cset and doing it > very piecemeal as I'm doing will help bisecting any subtle bug this may > introduce. > > > allow to parallelize BTF generation, where each CU would proceed in > > parallel generating local BTF, and then the final pass would merge and > > dedup BTFs. Currently reading and processing DWARF is the slowest part > > of the DWARF-to-BTF conversion, parallelization and maybe some other > > optimization seems like the only way to speed the process up. > > > > Acked-by: Andrii Nakryiko <andrii@xxxxxxxxxx> > > Thanks! > > - Arnaldo
