On Wed, Oct 30, 2024 at 6:01 PM Sami Tolvanen <samitolvanen@xxxxxxxxxx> wrote: > > Hi, > > Here's v5 of the DWARF modversions series. The main motivation is > modversions support for Rust, which is important for distributions > like Android that are about to ship Rust kernel modules. Per Luis' > request [1], v2 dropped the Rust specific bits from the series and > instead added the feature as an option for the entire kernel to > make it easier to evaluate the benefits of this approach, and to > get better test coverage. Matt is addressing Rust modversion_info > compatibility issues in a separate patch set [2] that depends on this > series, and actually allows modversions to be enabled with Rust. > > Short background: Unlike C, Rust source code doesn't have sufficient > information about the final ABI, as the compiler has considerable > freedom in adjusting structure layout, for example, which makes > using a source code parser like genksyms a non-starter. Based on > earlier feedback, this series uses DWARF debugging information for > computing versions. DWARF is an established and a relatively stable > format, which includes all the necessary ABI details, and adding a > CONFIG_DEBUG_INFO dependency for Rust symbol versioning seems like a > reasonable trade-off as most distributions already enable it. > > The first patch moves the genksyms CRC32 implementation to a shared > header file to avoid code duplication and the next 15 patches add > gendwarfksyms, a tool for computing symbol versions from DWARF. When > passed a list of exported symbols and object files, the tool > generates an expanded type string for each symbol and computes symbol > CRCs similarly to genksyms. gendwarfksyms is written in C and uses > libdw to process DWARF. Patch 17 ensures that debugging information > is present where we need it, patch 18 adds gendwarfksyms as an > alternative to genksyms, and the last patch adds documentation. > > v5 is based on v6.12-rc5 and for your convenience the series is also > available here: > > https://github.com/samitolvanen/linux/commits/gendwarfksyms-v5 > > If you also want to test the series with Rust modules, this branch > adds Matt's latest modversion_info series: > > https://github.com/samitolvanen/linux/commits/rustmodversions-v5 > > Sami > > > [1] https://lore.kernel.org/lkml/ZnIZEtkkQWEIGf9n@xxxxxxxxxxxxxxxxxxxxxx/ > [2] https://lore.kernel.org/lkml/20240925233854.90072-1-mmaurer@xxxxxxxxxx/ > > --- > > v5: > - Rebased on v6.12-rc5. > > - Fixed an issue with limiting structure expansion, and applied > Petr's clean-up. (Patch 10) > > - Dropped an unnecessary return statement in error path. (Patch > 12) > > - Addressed several other smaller issues Petr brought up. (Patches > 13, 14, and 15) > > - Added a KBUILD_GENDWARFKSYMS_STABLE flag to enable --stable for > the entire kernel build. (Patch 18) > Hi Sami, perfect timing: Nathan uploaded SLIM LLVM toolchain v19.1.3 KBUILD_GENDWARFKSYMS_STABLE is to be set manually? What value is recommended? Thanks. -Sedat- > - Updated documentation to include KBUILD flags. (Patch 19) > > - Picked up Reviewed-by tags from v4. > > v4: https://lore.kernel.org/lkml/20241008183823.36676-21-samitolvanen@xxxxxxxxxx/ > - Rebased on v6.12-rc2, which now includes all the prerequisites. > > - Dropped unnecessary name_only parameter for symbols.c::for_each > and cleaned up error handling. (Patch 3) > > - Fixed anonymous scope handling to ensure unnamed DIEs don't get > names. (Patch 4) > > - Added non-variant children to variant_type output, and included > DW_AT_discr_value attributes for variants. (Patch 9) > > - Added another symbol pointer test case. (Patch 16) > > - Picked up (Acked|Reviewed)-by tags from v3. > > > v3: https://lore.kernel.org/lkml/20240923181846.549877-22-samitolvanen@xxxxxxxxxx/ > - Updated SPX license headers. > > - Squashed the first two patches in v2 and tried to reduce churn as > much as reasonable. > > - Dropped patch 18 from v2 ("x86/asm-prototypes: Include > <asm/ptrace.h>") as it's addressed by a separate patch. > > - Changed the error handling code to immediately terminate instead > of propagating the errors back to main, which cleaned up the code > quite a bit. > > - Switched to the list and hashtable implementations in scripts and > dropped the remaining tools/include dependencies. Added a couple > missing list macros. (patch 1) > > - Moved the genksyms CRC32 implementation to scripts/include and > dropped the duplicate code. (patches 2 and 14) > > - Switched from ad-hoc command line parsing to getopt_long (patch 3). > > - Added structure member and function parameter names to the DIE > output to match genksyms behavior, and tweaked the symtypes format > to be more parser-friendly in general based on Petr's suggestions. > > - Replaced the declaration-only struct annotations with more generic > kABI stability rules that allow source code annotations to be used > where #ifndef __GENKSYMS__ was previously used. Added support for > rules that can be used to exclude enumerators from versioning. > (patch 16) > > - Per Miroslav's suggestion, added an option to hide structure > members from versioning when they're added to existing alignment > holes, for example. (patch 16) > > - Per Greg's request, added documentation and example macros for the > --stable features, and a couple of test cases. (patches 15, 16, and > 20) > > - Fixed making symtypes files, which need to depend on .o files with > gendwarfksyms. (patch 19) > > - Addressed several other smaller issues that Petr and Masahiro > kindly pointed out during the v2 review. > > > v2: https://lore.kernel.org/lkml/20240815173903.4172139-21-samitolvanen@xxxxxxxxxx/ > - Per Luis' request, dropped Rust-specific patches and added > gendwarfksyms as an alternative to genksyms for the entire > kernel. > > - Added support for missing DWARF features needed to handle > also non-Rust code. > > - Changed symbol address matching to use the symbol table > information instead of relying on addresses in DWARF. > > - Added __gendwarfksyms_ptr patches to ensure the compiler emits > the necessary type information in DWARF even for symbols that > are defined in other TUs. > > - Refactored debugging output and moved the more verbose output > behind --dump* flags. > > - Added a --symtypes flag for generating a genksyms-style > symtypes output based on Petr's feedback, and refactored > symbol version calculations to be based on symtypes instead > of raw --dump-dies output. > > - Based on feedback from Greg and Petr, added --stable flag and > support for reserved data structure fields and declaration-onl > structures. Also added examples for using these features. > > - Added a GENDWARFKSYMS option and hooked up kbuild support > for both C and assembly code. Note that with gendwarfksyms, > we have to actually build a temporary .o file for calculating > assembly modversions. > > v1: https://lore.kernel.org/lkml/20240617175818.58219-17-samitolvanen@xxxxxxxxxx/ > > --- > > Sami Tolvanen (19): > scripts: move genksyms crc32 implementation to a common include > tools: Add gendwarfksyms > gendwarfksyms: Add address matching > gendwarfksyms: Expand base_type > gendwarfksyms: Add a cache for processed DIEs > gendwarfksyms: Expand type modifiers and typedefs > gendwarfksyms: Expand subroutine_type > gendwarfksyms: Expand array_type > gendwarfksyms: Expand structure types > gendwarfksyms: Limit structure expansion > gendwarfksyms: Add die_map debugging > gendwarfksyms: Add symtypes output > gendwarfksyms: Add symbol versioning > gendwarfksyms: Add support for kABI rules > gendwarfksyms: Add support for reserved and ignored fields > gendwarfksyms: Add support for symbol type pointers > export: Add __gendwarfksyms_ptr_ references to exported symbols > kbuild: Add gendwarfksyms as an alternative to genksyms > Documentation/kbuild: Add DWARF module versioning > > Documentation/kbuild/gendwarfksyms.rst | 276 +++++ > Documentation/kbuild/index.rst | 1 + > include/linux/export.h | 15 + > kernel/module/Kconfig | 31 + > scripts/Makefile | 3 +- > scripts/Makefile.build | 41 +- > scripts/gendwarfksyms/.gitignore | 2 + > scripts/gendwarfksyms/Makefile | 12 + > scripts/gendwarfksyms/cache.c | 44 + > scripts/gendwarfksyms/die.c | 166 +++ > scripts/gendwarfksyms/dwarf.c | 1109 +++++++++++++++++++ > scripts/gendwarfksyms/examples/kabi.h | 141 +++ > scripts/gendwarfksyms/examples/kabi_ex0.c | 86 ++ > scripts/gendwarfksyms/examples/kabi_ex1.c | 89 ++ > scripts/gendwarfksyms/examples/kabi_ex2.c | 98 ++ > scripts/gendwarfksyms/examples/kabi_rules.c | 56 + > scripts/gendwarfksyms/examples/symbolptr.c | 33 + > scripts/gendwarfksyms/gendwarfksyms.c | 185 ++++ > scripts/gendwarfksyms/gendwarfksyms.h | 288 +++++ > scripts/gendwarfksyms/kabi.c | 263 +++++ > scripts/gendwarfksyms/symbols.c | 339 ++++++ > scripts/gendwarfksyms/types.c | 477 ++++++++ > scripts/genksyms/genksyms.c | 77 +- > scripts/include/crc32.h | 93 ++ > 24 files changed, 3840 insertions(+), 85 deletions(-) > create mode 100644 Documentation/kbuild/gendwarfksyms.rst > create mode 100644 scripts/gendwarfksyms/.gitignore > create mode 100644 scripts/gendwarfksyms/Makefile > create mode 100644 scripts/gendwarfksyms/cache.c > create mode 100644 scripts/gendwarfksyms/die.c > create mode 100644 scripts/gendwarfksyms/dwarf.c > create mode 100644 scripts/gendwarfksyms/examples/kabi.h > create mode 100644 scripts/gendwarfksyms/examples/kabi_ex0.c > create mode 100644 scripts/gendwarfksyms/examples/kabi_ex1.c > create mode 100644 scripts/gendwarfksyms/examples/kabi_ex2.c > create mode 100644 scripts/gendwarfksyms/examples/kabi_rules.c > create mode 100644 scripts/gendwarfksyms/examples/symbolptr.c > create mode 100644 scripts/gendwarfksyms/gendwarfksyms.c > create mode 100644 scripts/gendwarfksyms/gendwarfksyms.h > create mode 100644 scripts/gendwarfksyms/kabi.c > create mode 100644 scripts/gendwarfksyms/symbols.c > create mode 100644 scripts/gendwarfksyms/types.c > create mode 100644 scripts/include/crc32.h > > > base-commit: 81983758430957d9a5cb3333fe324fd70cf63e7e > -- > 2.47.0.163.g1226f6d8fa-goog >
