On Thu, Nov 21, 2024 at 9:42 PM Sami Tolvanen <samitolvanen@xxxxxxxxxx> wrote: > > Hi, > > Here's v6 of the DWARF modversions series. The main motivation is > modversions support for Rust, which is important for distributions > like Android that are about to ship Rust kernel modules. Per Luis' > request [1], v2 dropped the Rust specific bits from the series and > instead added the feature as an option for the entire kernel to > make it easier to evaluate the benefits of this approach, and to > get better test coverage. Matt is addressing Rust modversion_info > compatibility issues in a separate patch set [2] that depends on > this series, and actually allows modversions to be enabled with > Rust. > > Short background: Unlike C, Rust source code doesn't have sufficient > information about the final ABI, as the compiler has considerable > freedom in adjusting structure layout, for example, which makes > using a source code parser like genksyms a non-starter. Based on > earlier feedback, this series uses DWARF debugging information for > computing versions. DWARF is an established and a relatively stable > format, which includes all the necessary ABI details, and adding a > CONFIG_DEBUG_INFO dependency for Rust symbol versioning seems like a > reasonable trade-off as most distributions already enable it. > > The first 15 patches add gendwarfksyms, a tool for computing symbol > versions from DWARF. When passed a list of exported symbols and > object files, the tool generates an expanded type string for each > symbol and computes symbol versions. gendwarfksyms is written in C, > uses libdw to process DWARF, and zlib for CRC32. Patch 16 ensures > that debugging information is present where we need it, patch 17 > adds gendwarfksyms as an alternative to genksyms, and the last patch > adds documentation. > > v6 has changes to structure expansion and the kABI stability > features based on our backtesting results with previous Android > release kernels (see the change log below). It's also based on > linux-kbuild/for-next to include symtypes build rule clean-ups from > Masahiro [3]. For your convenience, the series is also available > here: > > https://github.com/samitolvanen/linux/commits/gendwarfksyms-v6 > Thanks for the update, Sami. What are your plans to get this upstream? Is Linux 6.13 the new development base? Personally, I would like to see a Linux v6.12 LTS version offered. LTO is not supported - might be worth mentioning this in the documentation patch with some explanations? BTW, I am testing with the latest kmod-git and pahole-git. I will give this a try when Linux v6.12.1 is released. Best regards, -Sedat- > If you also want to test the series with actual Rust modules, this > branch adds Matt's latest modversion_info series: > > https://github.com/samitolvanen/linux/commits/rustmodversions-v6 > > Sami > > > [1] https://lore.kernel.org/lkml/ZnIZEtkkQWEIGf9n@xxxxxxxxxxxxxxxxxxxxxx/ > [2] https://lore.kernel.org/linux-modules/20241030-extended-modversions-v8-0-93acdef62ce8@xxxxxxxxxx/ > [3] https://lore.kernel.org/linux-modules/CAK7LNAR9c+EEsOvPPn4qSq3gAFskYOXVd=dg8O+bKeeC-HMifw@xxxxxxxxxxxxxx/ > > --- > > v6: > - Dropped pointer expansion limits as this affects version > stability when exported symbols are removed. (Patch 9) > > - Changed local type definitions (in .c files) that are opaque > to external users to be treated as declarations even if a > definition is available. (Patch 9) > > - Switched to zlib's CRC32 implementation per Masahiro's > suggestion. (Patch 12) > > - Renamed struct_declonly kABI rule to simply declonly, as it > applies also to unions and enums, added a new rule for > overriding enumerator values, and refactored the examples. > (Patch 13) > > - Added --stable support for renamed structure members and > also added examples. (Patch 14) > > - Rebased on linux-kbuild/for-next for Masahiro's symtypes > build rule clean-ups. (Patch 17) > > - Updated the documentation reflect --stable changes. (Patch 18) > > v5: https://lore.kernel.org/lkml/20241030170106.1501763-21-samitolvanen@xxxxxxxxxx/ > - Rebased on v6.12-rc5. > > - Fixed an issue with limiting structure expansion, and applied > Petr's clean-up. (Patch 10) > > - Dropped an unnecessary return statement in error path. (Patch > 12) > > - Addressed several other smaller issues Petr brought up. (Patches > 13, 14, and 15) > > - Added a KBUILD_GENDWARFKSYMS_STABLE flag to enable --stable for > the entire kernel build. (Patch 18) > > - Updated documentation to include KBUILD flags. (Patch 19) > > - Picked up Reviewed-by tags from v4. > > v4: https://lore.kernel.org/lkml/20241008183823.36676-21-samitolvanen@xxxxxxxxxx/ > - Rebased on v6.12-rc2, which now includes all the prerequisites. > > - Dropped unnecessary name_only parameter for symbols.c::for_each > and cleaned up error handling. (Patch 3) > > - Fixed anonymous scope handling to ensure unnamed DIEs don't get > names. (Patch 4) > > - Added non-variant children to variant_type output, and included > DW_AT_discr_value attributes for variants. (Patch 9) > > - Added another symbol pointer test case. (Patch 16) > > - Picked up (Acked|Reviewed)-by tags from v3. > > v3: https://lore.kernel.org/lkml/20240923181846.549877-22-samitolvanen@xxxxxxxxxx/ > - Updated SPX license headers. > > - Squashed the first two patches in v2 and tried to reduce churn as > much as reasonable. > > - Dropped patch 18 from v2 ("x86/asm-prototypes: Include > <asm/ptrace.h>") as it's addressed by a separate patch. > > - Changed the error handling code to immediately terminate instead > of propagating the errors back to main, which cleaned up the code > quite a bit. > > - Switched to the list and hashtable implementations in scripts and > dropped the remaining tools/include dependencies. Added a couple > missing list macros. (patch 1) > > - Moved the genksyms CRC32 implementation to scripts/include and > dropped the duplicate code. (patches 2 and 14) > > - Switched from ad-hoc command line parsing to getopt_long (patch 3). > > - Added structure member and function parameter names to the DIE > output to match genksyms behavior, and tweaked the symtypes format > to be more parser-friendly in general based on Petr's suggestions. > > - Replaced the declaration-only struct annotations with more generic > kABI stability rules that allow source code annotations to be used > where #ifndef __GENKSYMS__ was previously used. Added support for > rules that can be used to exclude enumerators from versioning. > (patch 16) > > - Per Miroslav's suggestion, added an option to hide structure > members from versioning when they're added to existing alignment > holes, for example. (patch 16) > > - Per Greg's request, added documentation and example macros for the > --stable features, and a couple of test cases. (patches 15, 16, and > 20) > > - Fixed making symtypes files, which need to depend on .o files with > gendwarfksyms. (patch 19) > > - Addressed several other smaller issues that Petr and Masahiro > kindly pointed out during the v2 review. > > v2: https://lore.kernel.org/lkml/20240815173903.4172139-21-samitolvanen@xxxxxxxxxx/ > - Per Luis' request, dropped Rust-specific patches and added > gendwarfksyms as an alternative to genksyms for the entire > kernel. > > - Added support for missing DWARF features needed to handle > also non-Rust code. > > - Changed symbol address matching to use the symbol table > information instead of relying on addresses in DWARF. > > - Added __gendwarfksyms_ptr patches to ensure the compiler emits > the necessary type information in DWARF even for symbols that > are defined in other TUs. > > - Refactored debugging output and moved the more verbose output > behind --dump* flags. > > - Added a --symtypes flag for generating a genksyms-style > symtypes output based on Petr's feedback, and refactored > symbol version calculations to be based on symtypes instead > of raw --dump-dies output. > > - Based on feedback from Greg and Petr, added --stable flag and > support for reserved data structure fields and declaration-onl > structures. Also added examples for using these features. > > - Added a GENDWARFKSYMS option and hooked up kbuild support > for both C and assembly code. Note that with gendwarfksyms, > we have to actually build a temporary .o file for calculating > assembly modversions. > > v1: https://lore.kernel.org/lkml/20240617175818.58219-17-samitolvanen@xxxxxxxxxx/ > > --- > > Sami Tolvanen (18): > tools: Add gendwarfksyms > gendwarfksyms: Add address matching > gendwarfksyms: Expand base_type > gendwarfksyms: Add a cache for processed DIEs > gendwarfksyms: Expand type modifiers and typedefs > gendwarfksyms: Expand subroutine_type > gendwarfksyms: Expand array_type > gendwarfksyms: Expand structure types > gendwarfksyms: Limit structure expansion > gendwarfksyms: Add die_map debugging > gendwarfksyms: Add symtypes output > gendwarfksyms: Add symbol versioning > gendwarfksyms: Add support for kABI rules > gendwarfksyms: Add support for reserved and ignored fields > gendwarfksyms: Add support for symbol type pointers > export: Add __gendwarfksyms_ptr_ references to exported symbols > kbuild: Add gendwarfksyms as an alternative to genksyms > Documentation/kbuild: Add DWARF module versioning > > Documentation/kbuild/gendwarfksyms.rst | 308 ++++++ > Documentation/kbuild/index.rst | 1 + > include/linux/export.h | 15 + > kernel/module/Kconfig | 31 + > scripts/Makefile | 3 +- > scripts/Makefile.build | 35 +- > scripts/gendwarfksyms/.gitignore | 2 + > scripts/gendwarfksyms/Makefile | 12 + > scripts/gendwarfksyms/cache.c | 51 + > scripts/gendwarfksyms/die.c | 166 +++ > scripts/gendwarfksyms/dwarf.c | 1158 ++++++++++++++++++++ > scripts/gendwarfksyms/examples/kabi.h | 157 +++ > scripts/gendwarfksyms/examples/kabi_ex.c | 30 + > scripts/gendwarfksyms/examples/kabi_ex.h | 263 +++++ > scripts/gendwarfksyms/examples/symbolptr.c | 33 + > scripts/gendwarfksyms/gendwarfksyms.c | 185 ++++ > scripts/gendwarfksyms/gendwarfksyms.h | 301 +++++ > scripts/gendwarfksyms/kabi.c | 333 ++++++ > scripts/gendwarfksyms/symbols.c | 339 ++++++ > scripts/gendwarfksyms/types.c | 477 ++++++++ > 20 files changed, 3893 insertions(+), 7 deletions(-) > create mode 100644 Documentation/kbuild/gendwarfksyms.rst > create mode 100644 scripts/gendwarfksyms/.gitignore > create mode 100644 scripts/gendwarfksyms/Makefile > create mode 100644 scripts/gendwarfksyms/cache.c > create mode 100644 scripts/gendwarfksyms/die.c > create mode 100644 scripts/gendwarfksyms/dwarf.c > create mode 100644 scripts/gendwarfksyms/examples/kabi.h > create mode 100644 scripts/gendwarfksyms/examples/kabi_ex.c > create mode 100644 scripts/gendwarfksyms/examples/kabi_ex.h > create mode 100644 scripts/gendwarfksyms/examples/symbolptr.c > create mode 100644 scripts/gendwarfksyms/gendwarfksyms.c > create mode 100644 scripts/gendwarfksyms/gendwarfksyms.h > create mode 100644 scripts/gendwarfksyms/kabi.c > create mode 100644 scripts/gendwarfksyms/symbols.c > create mode 100644 scripts/gendwarfksyms/types.c > > > base-commit: 3596c721c4348b2a964e43f9296a0c01509ba927 > -- > 2.47.0.371.ga323438b13-goog >
