On Mon, Sep 23, 2024 at 2:19 PM Sami Tolvanen <samitolvanen@xxxxxxxxxx> wrote: > > Hi, > > Here's v3 of the DWARF modversions series [1][2]. The main > motivation remains modversions support for Rust, which is important > for distributions like Android that are eager to ship Rust kernel > modules. Per Luis' request [3], v2 dropped the Rust specific bits > from the series and instead added the feature as an option for > the entire kernel. Matt is still addressing Rust modversion_info > compatibility issues in a separate series [4], and we'll follow up > with a patch to actually allow CONFIG_MODVERSIONS with Rust once > everything else has been sorted out. > > A short background recap: Unlike C, Rust source code doesn't > have sufficient information about the final ABI, as the compiler > has considerable freedom in adjusting structure layout, for > example, which makes using a source code parser like genksyms a > non-starter. Based on Matt's suggestion and previous feedback from > maintainers, this series uses DWARF debugging information for > computing versions. DWARF is an established and a relatively stable > format, which includes all the necessary ABI details, and adding a > CONFIG_DEBUG_INFO dependency for Rust symbol versioning seems like a > reasonable trade-off. > > The first two patches add more list macros to scripts/include and > move the genksyms CRC32 implementation to a shared header file. The > next 15 patches add gendwarfksyms, a tool for computing symbol > versions from DWARF. When passed a list of exported symbols and > object files, the tool generates an expanded type string for each > symbol and computes symbol CRCs similarly to genksyms. gendwarfksyms > is written in C and uses libdw to process DWARF. Patch 18 ensures > that debugging information is present where we need it, patch 19 > adds gendwarfksyms as an alternative to genksyms, and the last patch > adds documentation. > > Note that v3 is based on next-20240923 as it depends on Masahiro's > scripts/include changes. For x86, we also need a separate small > patch to include asm/ptrace.h in asm/ftrace.h. [5] For your > convenience, you can find this series with all the prerequisites > here: > > https://github.com/samitolvanen/linux/commits/gendwarfksyms-v3 > > If you also want to test the series with Rust modules, this branch > adds Matt's modversion_info series and a small patch to enable Rust > modversions: > > https://github.com/samitolvanen/linux/commits/rustmodversions-v3 > > Looking forward to hearing your thoughts! > > Sami > > > [1] v1: https://lore.kernel.org/lkml/20240617175818.58219-17-samitolvanen@xxxxxxxxxx/ > [2] v2: https://lore.kernel.org/lkml/20240815173903.4172139-21-samitolvanen@xxxxxxxxxx/ > [3] https://lore.kernel.org/lkml/ZnIZEtkkQWEIGf9n@xxxxxxxxxxxxxxxxxxxxxx/ > [4] https://lore.kernel.org/lkml/20240806212106.617164-1-mmaurer@xxxxxxxxxx/ > [5] https://lore.kernel.org/lkml/20240916221557.846853-2-samitolvanen@xxxxxxxxxx/ > > --- > > Changes in v3: > - Updated SPX license headers. > > - Squashed the first two patches in v2 and tried to reduce churn as > much as reasonable. > > - Dropped patch 18 from v2 ("x86/asm-prototypes: Include > <asm/ptrace.h>") as it's addressed by a separate patch. [5] > > - Changed the error handling code to immediately terminate instead > of propagating the errors back to main, which cleaned up the code > quite a bit. > > - Switched to the list and hashtable implementations in scripts and > dropped the remaining tools/include dependencies. Added a couple > missing list macros. (patch 1) > > - Moved the genksyms CRC32 implementation to scripts/include and > dropped the duplicate code. (patches 2 and 14) > > - Switched from ad-hoc command line parsing to getopt_long (patch 3). > > - Added structure member and function parameter names to the DIE > output to match genksyms behavior, and tweaked the symtypes format > to be more parser-friendly in general based on Petr's suggestions. > > - Replaced the declaration-only struct annotations with more generic > kABI stability rules that allow source code annotations to be used > where #ifndef __GENKSYMS__ was previously used. Added support for > rules that can be used to exclude enumerators from versioning. > (patch 16) > > - Per Miroslav's suggestion, added an option to hide structure > members from versioning when they're added to existing alignment > holes, for example. (patch 16) > > - Per Greg's request, added documentation and example macros for the > --stable features, and a couple of test cases. (patches 15, 16, and > 20) > > - Fixed making symtypes files, which need to depend on .o files with > gendwarfksyms. (patch 19) > > - Addressed several other smaller issues that Petr and Masahiro > kindly pointed out during the v2 review. > > Changes in v2: > - Per Luis' request, dropped Rust-specific patches and added > gendwarfksyms as an alternative to genksyms for the entire > kernel. > > - Added support for missing DWARF features needed to handle > also non-Rust code. > > - Changed symbol address matching to use the symbol table > information instead of relying on addresses in DWARF. > > - Added __gendwarfksyms_ptr patches to ensure the compiler emits > the necessary type information in DWARF even for symbols that > are defined in other TUs. > > - Refactored debugging output and moved the more verbose output > behind --dump* flags. > > - Added a --symtypes flag for generating a genksyms-style > symtypes output based on Petr's feedback, and refactored > symbol version calculations to be based on symtypes instead > of raw --dump-dies output. > > - Based on feedback from Greg and Petr, added --stable flag and > support for reserved data structure fields and declaration-onl > structures. Also added examples for using these features. > > - Added a GENDWARFKSYMS option and hooked up kbuild support > for both C and assembly code. Note that with gendwarfksyms, > we have to actually build a temporary .o file for calculating > assembly modversions. > > --- > > Sami Tolvanen (20): > scripts: import more list macros > scripts: move genksyms crc32 implementation to a common include > tools: Add gendwarfksyms > gendwarfksyms: Add address matching > gendwarfksyms: Expand base_type > gendwarfksyms: Add a cache for processed DIEs > gendwarfksyms: Expand type modifiers and typedefs > gendwarfksyms: Expand subroutine_type > gendwarfksyms: Expand array_type > gendwarfksyms: Expand structure types > gendwarfksyms: Limit structure expansion > gendwarfksyms: Add die_map debugging > gendwarfksyms: Add symtypes output > gendwarfksyms: Add symbol versioning > gendwarfksyms: Add support for kABI rules > gendwarfksyms: Add support for reserved and ignored fields > gendwarfksyms: Add support for symbol type pointers > export: Add __gendwarfksyms_ptr_ references to exported symbols > kbuild: Add gendwarfksyms as an alternative to genksyms > Documentation/kbuild: Add DWARF module versioning > > Documentation/kbuild/gendwarfksyms.rst | 274 +++++ > Documentation/kbuild/index.rst | 1 + > include/linux/export.h | 15 + > kernel/module/Kconfig | 31 + > scripts/Makefile | 3 +- > scripts/Makefile.build | 39 +- > scripts/gendwarfksyms/.gitignore | 2 + > scripts/gendwarfksyms/Makefile | 12 + > scripts/gendwarfksyms/cache.c | 44 + > scripts/gendwarfksyms/die.c | 166 +++ > scripts/gendwarfksyms/dwarf.c | 1085 +++++++++++++++++++ > scripts/gendwarfksyms/examples/kabi.h | 141 +++ > scripts/gendwarfksyms/examples/kabi_ex0.c | 86 ++ > scripts/gendwarfksyms/examples/kabi_ex1.c | 89 ++ > scripts/gendwarfksyms/examples/kabi_ex2.c | 98 ++ > scripts/gendwarfksyms/examples/kabi_rules.c | 56 + > scripts/gendwarfksyms/examples/symbolptr.c | 29 + > scripts/gendwarfksyms/gendwarfksyms.c | 195 ++++ > scripts/gendwarfksyms/gendwarfksyms.h | 351 ++++++ > scripts/gendwarfksyms/kabi.c | 214 ++++ > scripts/gendwarfksyms/symbols.c | 317 ++++++ > scripts/gendwarfksyms/types.c | 477 ++++++++ > scripts/genksyms/genksyms.c | 77 +- > scripts/include/crc32.h | 93 ++ > scripts/include/list.h | 50 + > 25 files changed, 3860 insertions(+), 85 deletions(-) > create mode 100644 Documentation/kbuild/gendwarfksyms.rst > create mode 100644 scripts/gendwarfksyms/.gitignore > create mode 100644 scripts/gendwarfksyms/Makefile > create mode 100644 scripts/gendwarfksyms/cache.c > create mode 100644 scripts/gendwarfksyms/die.c > create mode 100644 scripts/gendwarfksyms/dwarf.c > create mode 100644 scripts/gendwarfksyms/examples/kabi.h > create mode 100644 scripts/gendwarfksyms/examples/kabi_ex0.c > create mode 100644 scripts/gendwarfksyms/examples/kabi_ex1.c > create mode 100644 scripts/gendwarfksyms/examples/kabi_ex2.c > create mode 100644 scripts/gendwarfksyms/examples/kabi_rules.c > create mode 100644 scripts/gendwarfksyms/examples/symbolptr.c > create mode 100644 scripts/gendwarfksyms/gendwarfksyms.c > create mode 100644 scripts/gendwarfksyms/gendwarfksyms.h > create mode 100644 scripts/gendwarfksyms/kabi.c > create mode 100644 scripts/gendwarfksyms/symbols.c > create mode 100644 scripts/gendwarfksyms/types.c > create mode 100644 scripts/include/crc32.h > > > base-commit: ef545bc03a65438cabe87beb1b9a15b0ffcb6ace > -- > 2.46.0.792.g87dc391469-goog > This patch set looks fairly reasonable. Acked-by: Neal Gompa <neal@xxxxxxxxx> -- 真実はいつも一つ！/ Always, there's only one truth!
