On Thu, Aug 15, 2024 at 7:39 PM Sami Tolvanen <samitolvanen@xxxxxxxxxx> wrote: > > Hi, > > Here's v2 of the DWARF modversions series [1]. The main motivation > remains modversions support for Rust, which is important for > distributions like Android that are eager to ship Rust kernel > modules. However, per Luis' request [2], v2 drops all Rust specific > bits from the series and instead adds the feature as an option > for the entire kernel. Matt is addressing Rust modversion_info > compatibility issues in a separate series [3], and we'll follow up > with a patch to actually allow CONFIG_MODVERSIONS with Rust once > these have been sorted out. > > A short background recap: Unlike C, Rust source code doesn't have > sufficient information about the final ABI, as the compiler has > considerable freedom in adjusting structure layout for improved > performance [4], for example, which makes using a source code > parser like genksyms a non-starter. Based on Matt's suggestion and > previous feedback from maintainers, this series uses DWARF debugging > information for computing versions. DWARF is an established and > a relatively stable format, which includes all the necessary ABI > details, and adding a CONFIG_DEBUG_INFO dependency for Rust symbol > versioning seems like a reasonable trade-off. > > The first 16 patches of this series add a small tool for computing > symbol versions from DWARF, called gendwarfksyms. When passed a > list of exported symbols and an object file, the tool generates > an expanded type string for each symbol, and computes symbol CRCs > similarly to genksyms. gendwarfksyms is written in C and uses libdw > to process DWARF, mainly because of the existing support for C host > tools that use elfutils (e.g., objtool). The next two patches ensure > that debugging information is present where we need it and fix a > compilation issue with x86 asm-prototypes.h. The last patch adds > gendwarfksyms as an alternative to genksyms. > > A quick note about performance: On my development system, building > x86_64 defconfig with MODVERSIONS takes about 59.4s with gcc 13 > (avg. of ten runs). Adding DEBUG_INFO_DWARF5 increases the build > time by ~23% to 73.3s. Switching from GENKSYMS to GENDWARFKSYMS > reduces the build time by 6% to 68.9s, which is still ~16% slower > than genksyms without debugging information. Therefore, if you > already build kernels with debugging information, gendwarfksyms > should be slightly faster. YMMV, of course. > > Things would change with LTO, because we won't have full DWARF > until we have an ELF binary, which means we'd have to process > vmlinux.o. This version of gendwarfksyms is still single-threaded > as it seems we can't rely on libdw to be thread-safe. Processing > a ThinLTO x86_64 defconfig vmlinux.o on my system takes ~2m16s, > and would have to happen even on incremental builds, just like > LTO linking itself. As cross-language LTO presumably isn't wildly > popular yet, gendwarfksyms intentionally depends in !LTO in this > version. > > Looking forward to hearing your thoughts! > Hi Sami, so this work is on top of Linux v6.11-rc3 - can you tag it as gendwarfksyms-v2? Thanks. Best regards, -Sedat- https://github.com/samitolvanen/linux/tree/gendwarfksyms https://github.com/samitolvanen/linux/tags > Sami > > [1] https://lore.kernel.org/lkml/20240617175818.58219-17-samitolvanen@xxxxxxxxxx/ > [2] https://lore.kernel.org/lkml/ZnIZEtkkQWEIGf9n@xxxxxxxxxxxxxxxxxxxxxx/ > [3] https://lore.kernel.org/lkml/20240806212106.617164-1-mmaurer@xxxxxxxxxx/ > [4] https://lore.kernel.org/rust-for-linux/CAGSQo005hRiUZdeppCifDqG9zFDJRwahpBLE4x7-MyfJscn7tQ@xxxxxxxxxxxxxx/ > > --- > > Changes in v2: > - Per Luis' request, dropped Rust-specific patches and added > gendwarfksyms as an alternative to genksyms for the entire > kernel. > > - Added support for missing DWARF features needed to handle > also non-Rust code. > > - Changed symbol address matching to use the symbol table > information instead of relying on addresses in DWARF. > > - Added __gendwarfksyms_ptr patches to ensure the compiler emits > the necessary type information in DWARF even for symbols that > are defined in other TUs. > > - Refactored debugging output and moved the more verbose output > behind --dump* flags. > > - Added a --symtypes flag for generating a genksyms-style > symtypes output based on Petr's feedback, and refactored > symbol version calculations to be based on symtypes instead > of raw --dump-dies output. > > - Based on feedback from Greg and Petr, added --stable flag and > support for reserved data structure fields and declaration-onl > structures. Also added examples for using these features. > > - Added a GENDWARFKSYMS option and hooked up kbuild support > for both C and assembly code. Note that with gendwarfksyms, > we have to actually build a temporary .o file for calculating > assembly modversions. > > --- > > Sami Tolvanen (19): > tools: Add gendwarfksyms > gendwarfksyms: Add symbol list handling > gendwarfksyms: Add address matching > gendwarfksyms: Add support for type pointers > gendwarfksyms: Expand base_type > gendwarfksyms: Add a cache for processed DIEs > gendwarfksyms: Expand type modifiers and typedefs > gendwarfksyms: Expand subroutine_type > gendwarfksyms: Expand array_type > gendwarfksyms: Expand structure types > gendwarfksyms: Limit structure expansion > gendwarfksyms: Add die_map debugging > gendwarfksyms: Add symtypes output > gendwarfksyms: Add symbol versioning > gendwarfksyms: Add support for declaration-only data structures > gendwarfksyms: Add support for reserved structure fields > export: Add __gendwarfksyms_ptr_ references to exported symbols > x86/asm-prototypes: Include <asm/ptrace.h> > kbuild: Add gendwarfksyms as an alternative to genksyms > > arch/x86/include/asm/asm-prototypes.h | 1 + > include/linux/export.h | 15 + > kernel/module/Kconfig | 31 + > scripts/Makefile | 3 +- > scripts/Makefile.build | 34 +- > scripts/gendwarfksyms/.gitignore | 2 + > scripts/gendwarfksyms/Makefile | 12 + > scripts/gendwarfksyms/cache.c | 51 ++ > scripts/gendwarfksyms/crc32.c | 69 ++ > scripts/gendwarfksyms/crc32.h | 34 + > scripts/gendwarfksyms/die.c | 196 +++++ > scripts/gendwarfksyms/dwarf.c | 973 ++++++++++++++++++++++ > scripts/gendwarfksyms/examples/declonly.c | 31 + > scripts/gendwarfksyms/examples/reserved.c | 66 ++ > scripts/gendwarfksyms/gendwarfksyms.c | 201 +++++ > scripts/gendwarfksyms/gendwarfksyms.h | 275 ++++++ > scripts/gendwarfksyms/symbols.c | 392 +++++++++ > scripts/gendwarfksyms/types.c | 557 +++++++++++++ > 18 files changed, 2936 insertions(+), 7 deletions(-) > create mode 100644 scripts/gendwarfksyms/.gitignore > create mode 100644 scripts/gendwarfksyms/Makefile > create mode 100644 scripts/gendwarfksyms/cache.c > create mode 100644 scripts/gendwarfksyms/crc32.c > create mode 100644 scripts/gendwarfksyms/crc32.h > create mode 100644 scripts/gendwarfksyms/die.c > create mode 100644 scripts/gendwarfksyms/dwarf.c > create mode 100644 scripts/gendwarfksyms/examples/declonly.c > create mode 100644 scripts/gendwarfksyms/examples/reserved.c > create mode 100644 scripts/gendwarfksyms/gendwarfksyms.c > create mode 100644 scripts/gendwarfksyms/gendwarfksyms.h > create mode 100644 scripts/gendwarfksyms/symbols.c > create mode 100644 scripts/gendwarfksyms/types.c > > > base-commit: 7c626ce4bae1ac14f60076d00eafe71af30450ba > -- > 2.46.0.184.g6999bdac58-goog > >
