Re: [RFC PATCH dwarves] btf: add support for split BTF loading and encoding

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, Nov 5, 2020 at 3:42 AM Arnaldo Carvalho de Melo <acme@xxxxxxxxxx> wrote:
>
> Em Wed, Nov 04, 2020 at 08:39:36PM -0800, Andrii Nakryiko escreveu:
> > Add support for generating split BTF, in which there is a designated base
> > BTF, containing a base set of types, and a split BTF, which extends main BTF
> > with extra types, that can reference types and strings from the main BTF.
> >
> > This is going to be used to generate compact BTFs for kernel modules, with
> > vmlinux BTF being a main BTF, which all kernel modules are based off of.
> >
> > These changes rely on patch set [0] to be present in libbpf submodule.
> >
> >   [0] https://patchwork.kernel.org/project/netdevbpf/list/?series=377859&state=*
> >
> > Signed-off-by: Andrii Nakryiko <andrii@xxxxxxxxxx>
> > ---
> >
> > This is posted before libbpf changes landed to show end-to-end how kernel
> > module BTFs are going to be integrated into the kernel. Once libbpf split BTF
> > support lands, I'll sync it into Github repo and will post a proper v1.
> >
> >  btf_encoder.c | 15 ++++++++-------
> >  btf_loader.c  |  2 +-
> >  libbtf.c      | 43 +++++++++++++++++++++++++++----------------
> >  libbtf.h      |  4 +++-
> >  pahole.c      | 23 +++++++++++++++++++++++
> >  5 files changed, 62 insertions(+), 25 deletions(-)
> >
> > diff --git a/btf_encoder.c b/btf_encoder.c
> > index 4c92908beab2..d67e29b9cbee 100644
> > --- a/btf_encoder.c
> > +++ b/btf_encoder.c
> > @@ -12,6 +12,7 @@
> >  #include "dwarves.h"
> >  #include "libbtf.h"
> >  #include "lib/bpf/include/uapi/linux/btf.h"
> > +#include "lib/bpf/src/libbpf.h"
> >  #include "hash.h"
> >  #include "elf_symtab.h"
> >  #include "btf_encoder.h"
> > @@ -343,7 +344,7 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
> >       }
> >
> >       if (!btfe) {
> > -             btfe = btf_elf__new(cu->filename, cu->elf);
> > +             btfe = btf_elf__new(cu->filename, cu->elf, base_btf);
> >               if (!btfe)
> >                       return -1;
> >
> > @@ -358,22 +359,22 @@ int cu__encode_btf(struct cu *cu, int verbose, bool force,
> >                       printf("File %s:\n", btfe->filename);
> >       }
> >
> > +     btf_elf__verbose = verbose;
> > +     btf_elf__force = force;
> > +     type_id_off = btf__get_nr_types(btfe->btf);
> > +
> >       if (!has_index_type) {
> >               /* cu__find_base_type_by_name() takes "type_id_t *id" */
> >               type_id_t id;
> >               if (cu__find_base_type_by_name(cu, "int", &id)) {
> >                       has_index_type = true;
> > -                     array_index_id = id;
> > +                     array_index_id = type_id_off + id;
> >               } else {
> >                       has_index_type = false;
> > -                     array_index_id = cu->types_table.nr_entries;
> > +                     array_index_id = type_id_off + cu->types_table.nr_entries;
> >               }
> >       }
> >
> > -     btf_elf__verbose = verbose;
> > -     btf_elf__force = force;
> > -     type_id_off = btf__get_nr_types(btfe->btf);
> > -
> >       cu__for_each_type(cu, core_id, pos) {
> >               int32_t btf_type_id = tag__encode_btf(cu, pos, core_id, btfe, array_index_id, type_id_off);
> >
> > diff --git a/btf_loader.c b/btf_loader.c
> > index 6ea207ea65b4..ec286f413f36 100644
> > --- a/btf_loader.c
> > +++ b/btf_loader.c
> > @@ -534,7 +534,7 @@ struct debug_fmt_ops btf_elf__ops;
> >  int btf_elf__load_file(struct cus *cus, struct conf_load *conf, const char *filename)
> >  {
> >       int err;
> > -     struct btf_elf *btfe = btf_elf__new(filename, NULL);
> > +     struct btf_elf *btfe = btf_elf__new(filename, NULL, base_btf);
> >
> >       if (btfe == NULL)
> >               return -1;
> > diff --git a/libbtf.c b/libbtf.c
> > index babf4fe8cd9e..3c52aa0d482b 100644
> > --- a/libbtf.c
> > +++ b/libbtf.c
> > @@ -27,6 +27,7 @@
> >  #include "dwarves.h"
> >  #include "elf_symtab.h"
> >
> > +struct btf *base_btf;
> >  uint8_t btf_elf__verbose;
> >  uint8_t btf_elf__force;
> >
> > @@ -52,9 +53,9 @@ int btf_elf__load(struct btf_elf *btfe)
> >       /* free initial empty BTF */
> >       btf__free(btfe->btf);
> >       if (btfe->raw_btf)
> > -             btfe->btf = btf__parse_raw(btfe->filename);
> > +             btfe->btf = btf__parse_raw_split(btfe->filename, btfe->base_btf);
> >       else
> > -             btfe->btf = btf__parse_elf(btfe->filename, NULL);
> > +             btfe->btf = btf__parse_elf_split(btfe->filename, btfe->base_btf);
> >
> >       err = libbpf_get_error(btfe->btf);
> >       if (err)
> > @@ -63,7 +64,7 @@ int btf_elf__load(struct btf_elf *btfe)
> >       return 0;
> >  }
> >
> > -struct btf_elf *btf_elf__new(const char *filename, Elf *elf)
> > +struct btf_elf *btf_elf__new(const char *filename, Elf *elf, struct btf *base_btf)
> >  {
> >       struct btf_elf *btfe = zalloc(sizeof(*btfe));
> >       GElf_Shdr shdr;
> > @@ -77,7 +78,8 @@ struct btf_elf *btf_elf__new(const char *filename, Elf *elf)
> >       if (btfe->filename == NULL)
> >               goto errout;
> >
> > -     btfe->btf = btf__new_empty();
> > +     btfe->base_btf = base_btf;
> > +     btfe->btf = btf__new_empty_split(base_btf);
> >       if (libbpf_get_error(btfe->btf)) {
> >               fprintf(stderr, "%s: failed to create empty BTF.\n", __func__);
> >               goto errout;
> > @@ -679,11 +681,11 @@ static int btf_elf__write(const char *filename, struct btf *btf)
> >  {
> >       GElf_Shdr shdr_mem, *shdr;
> >       GElf_Ehdr ehdr_mem, *ehdr;
> > -     Elf_Data *btf_elf = NULL;
> > +     Elf_Data *btf_data = NULL;
>
> Can you please split this into two patches, one doing just the rename
> of btf_elf to btf_data and then moving to btf__new_empty_split()? Eases
> reviewing.

sure, will do in the next version

>
> With this split btf code would it be possible to paralelize the encoding
> of the modules BTF? I have to check the other patches and how this gets
> used in the kernel build process... :-)

Yes, each module's BTF is generated completely independently. See some
numbers in [0].

  [0] https://patchwork.kernel.org/project/netdevbpf/patch/20201105045140.2589346-4-andrii@xxxxxxxxxx/

>
> - Arnaldo
>

[...]



[Index of Archives]     [Linux Samsung SoC]     [Linux Rockchip SoC]     [Linux Actions SoC]     [Linux for Synopsys ARC Processors]     [Linux NFS]     [Linux NILFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]


  Powered by Linux