On Thu, Sep 19, 2024 at 07:08:42PM +0100, Sam James wrote: > Masahiro Yamada <masahiroy@xxxxxxxxxx> writes: > > > On Fri, Sep 20, 2024 at 2:07 AM Daniel Gomez <da.gomez@xxxxxxxxxxx> wrote: > >> > >> On Fri, Sep 06, 2024 at 10:45:03AM -0400, Kris Van Hees wrote: > >> > Create file module.builtin.ranges that can be used to find where > >> > built-in modules are located by their addresses. This will be useful for > >> > tracing tools to find what functions are for various built-in modules. > >> > > >> > The offset range data for builtin modules is generated using: > >> > - modules.builtin: associates object files with module names > >> > - vmlinux.map: provides load order of sections and offset of first member > >> > per section > >> > - vmlinux.o.map: provides offset of object file content per section > >> > - .*.cmd: build cmd file with KBUILD_MODFILE > >> > > >> > The generated data will look like: > >> > > >> > .text 00000000-00000000 = _text > >> > .text 0000baf0-0000cb10 amd_uncore > >> > .text 0009bd10-0009c8e0 iosf_mbi > >> > ... > >> > .text 00b9f080-00ba011a intel_skl_int3472_discrete > >> > .text 00ba0120-00ba03c0 intel_skl_int3472_discrete intel_skl_int3472_tps68470 > >> > .text 00ba03c0-00ba08d6 intel_skl_int3472_tps68470 > >> > ... > >> > .data 00000000-00000000 = _sdata > >> > .data 0000f020-0000f680 amd_uncore > >> > > >> > For each ELF section, it lists the offset of the first symbol. This can > >> > be used to determine the base address of the section at runtime. > >> > > >> > Next, it lists (in strict ascending order) offset ranges in that section > >> > that cover the symbols of one or more builtin modules. Multiple ranges > >> > can apply to a single module, and ranges can be shared between modules. > >> > > >> > The CONFIG_BUILTIN_MODULE_RANGES option controls whether offset range data > >> > is generated for kernel modules that are built into the kernel image. > >> > > >> > How it works: > >> > > >> > 1. The modules.builtin file is parsed to obtain a list of built-in > >> > module names and their associated object names (the .ko file that > >> > the module would be in if it were a loadable module, hereafter > >> > referred to as <kmodfile>). This object name can be used to > >> > identify objects in the kernel compile because any C or assembler > >> > code that ends up into a built-in module will have the option > >> > -DKBUILD_MODFILE=<kmodfile> present in its build command, and those > >> > can be found in the .<obj>.cmd file in the kernel build tree. > >> > > >> > If an object is part of multiple modules, they will all be listed > >> > in the KBUILD_MODFILE option argument. > >> > > >> > This allows us to conclusively determine whether an object in the > >> > kernel build belong to any modules, and which. > >> > > >> > 2. The vmlinux.map is parsed next to determine the base address of each > >> > top level section so that all addresses into the section can be > >> > turned into offsets. This makes it possible to handle sections > >> > getting loaded at different addresses at system boot. > >> > > >> > We also determine an 'anchor' symbol at the beginning of each > >> > section to make it possible to calculate the true base address of > >> > a section at runtime (i.e. symbol address - symbol offset). > >> > > >> > We collect start addresses of sections that are included in the top > >> > level section. This is used when vmlinux is linked using vmlinux.o, > >> > because in that case, we need to look at the vmlinux.o linker map to > >> > know what object a symbol is found in. > >> > > >> > And finally, we process each symbol that is listed in vmlinux.map > >> > (or vmlinux.o.map) based on the following structure: > >> > > >> > vmlinux linked from vmlinux.a: > >> > > >> > vmlinux.map: > >> > <top level section> > >> > <included section> -- might be same as top level section) > >> > <object> -- built-in association known > >> > <symbol> -- belongs to module(s) object belongs to > >> > ... > >> > > >> > vmlinux linked from vmlinux.o: > >> > > >> > vmlinux.map: > >> > <top level section> > >> > <included section> -- might be same as top level section) > >> > vmlinux.o -- need to use vmlinux.o.map > >> > <symbol> -- ignored > >> > ... > >> > > >> > vmlinux.o.map: > >> > <section> > >> > <object> -- built-in association known > >> > <symbol> -- belongs to module(s) object belongs to > >> > ... > >> > > >> > 3. As sections, objects, and symbols are processed, offset ranges are > >> > constructed in a straight-forward way: > >> > > >> > - If the symbol belongs to one or more built-in modules: > >> > - If we were working on the same module(s), extend the range > >> > to include this object > >> > - If we were working on another module(s), close that range, > >> > and start the new one > >> > - If the symbol does not belong to any built-in modules: > >> > - If we were working on a module(s) range, close that range > >> > > >> > Signed-off-by: Kris Van Hees <kris.van.hees@xxxxxxxxxx> > >> > Reviewed-by: Nick Alcock <nick.alcock@xxxxxxxxxx> > >> > Reviewed-by: Alan Maguire <alan.maguire@xxxxxxxxxx> > >> > Reviewed-by: Steven Rostedt (Google) <rostedt@xxxxxxxxxxx> > >> > Tested-by: Sam James <sam@xxxxxxxxxx> > >> > --- > >> > > >> > Notes: > >> > Changes since v9: > >> > - Reverted support for build directory as optional 4th argument. > >> > - Added modules.builtin.ranges and vmlinux.o.map to CLEAN_FILES. > >> > - Fixed support for sparc64. > >> > > >> > Changes since v8: > >> > - Added support for built-in Rust modules. > >> > - Added optional 4th argument to specify kernel build directory. > >> > > >> > Changes since v7: > >> > - Removed extra close(fn). > >> > - Make CONFIG_BUILTIN_MODULE_RANGES depend on !lTO. > >> > > >> > Changes since v6: > >> > - Applied Masahiro Yamada's suggestions (Kconfig, makefile, script). > >> > > >> > Changes since v5: > >> > - Removed unnecessary compatibility info from option description. > >> > > >> > Changes since v4: > >> > - Improved commit description to explain the why and how. > >> > - Documented dependency on GNU AWK for CONFIG_BUILTIN_MODULE_RANGES. > >> > - Improved comments in generate_builtin_ranges.awk > >> > - Improved logic in generate_builtin_ranges.awk to handle incorrect > >> > object size information in linker maps > >> > > >> > Changes since v3: > >> > - Consolidated patches 2 through 5 into a single patch > >> > - Move CONFIG_BUILTIN_MODULE_RANGES to Kconfig.debug > >> > - Make CONFIG_BUILTIN_MODULE_RANGES select CONFIG_VMLINUX_MAP > >> > - Disable CONFIG_BUILTIN_MODULE_RANGES if CONFIG_LTO_CLANG_(FULL|THIN)=y > >> > - Support LLVM (lld) compiles in generate_builtin_ranges.awk > >> > - Support CONFIG_LD_DEAD_CODE_DATA_ELIMINATION=y > >> > > >> > Changes since v2: > >> > - Add explicit dependency on FTRACE for CONFIG_BUILTIN_MODULE_RANGES > >> > - 1st arg to generate_builtin_ranges.awk is now modules.builtin.modinfo > >> > - Switched from using modules.builtin.objs to parsing .*.cmd files > >> > - Parse data from .*.cmd in generate_builtin_ranges.awk > >> > - Use $(real-prereqs) rather than $(filter-out ...) > >> > --- > >> > > >> > Documentation/process/changes.rst | 7 + > >> > Makefile | 1 + > >> > lib/Kconfig.debug | 15 + > >> > scripts/Makefile.vmlinux | 18 + > >> > scripts/Makefile.vmlinux_o | 3 + > >> > scripts/generate_builtin_ranges.awk | 508 ++++++++++++++++++++++++++++ > >> > 6 files changed, 552 insertions(+) > >> > create mode 100755 scripts/generate_builtin_ranges.awk > >> > > >> > diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst > >> > index 3fc63f27c226..00f1ed7c59c3 100644 > >> > --- a/Documentation/process/changes.rst > >> > +++ b/Documentation/process/changes.rst > >> > @@ -64,6 +64,7 @@ GNU tar 1.28 tar --version > >> > gtags (optional) 6.6.5 gtags --version > >> > mkimage (optional) 2017.01 mkimage --version > >> > Python (optional) 3.5.x python3 --version > >> > +GNU AWK (optional) 5.1.0 gawk --version > >> > ====================== =============== ======================================== > >> > > >> > .. [#f1] Sphinx is needed only to build the Kernel documentation > >> > @@ -192,6 +193,12 @@ platforms. The tool is available via the ``u-boot-tools`` package or can be > >> > built from the U-Boot source code. See the instructions at > >> > https://protect2.fireeye.com/v1/url?k=6b601b01-34fc322b-6b61904e-000babe598f7-59f65dfa7ee29fbf&q=1&e=8abd7076-5118-4660-a833-f762c2c71d32&u=https%3A%2F%2Fdocs.u-boot.org%2Fen%2Flatest%2Fbuild%2Ftools.html%23building-tools-for-linux > >> > > >> > +GNU AWK > >> > +------- > >> > + > >> > +GNU AWK is needed if you want kernel builds to generate address range data for > >> > +builtin modules (CONFIG_BUILTIN_MODULE_RANGES). > >> > + > >> > System utilities > >> > **************** > >> > > >> > diff --git a/Makefile b/Makefile > >> > index d57cfc6896b8..ec98a1e5b257 100644 > >> > --- a/Makefile > >> > +++ b/Makefile > >> > @@ -1482,6 +1482,7 @@ endif # CONFIG_MODULES > >> > # Directories & files removed with 'make clean' > >> > CLEAN_FILES += vmlinux.symvers modules-only.symvers \ > >> > modules.builtin modules.builtin.modinfo modules.nsdeps \ > >> > + modules.builtin.ranges vmlinux.o.map \ > >> > compile_commands.json rust/test \ > >> > rust-project.json .vmlinux.objs .vmlinux.export.c > >> > > >> > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug > >> > index a30c03a66172..5e2f30921cb2 100644 > >> > --- a/lib/Kconfig.debug > >> > +++ b/lib/Kconfig.debug > >> > @@ -571,6 +571,21 @@ config VMLINUX_MAP > >> > pieces of code get eliminated with > >> > CONFIG_LD_DEAD_CODE_DATA_ELIMINATION. > >> > > >> > +config BUILTIN_MODULE_RANGES > >> > + bool "Generate address range information for builtin modules" > >> > + depends on !LTO > >> > + depends on VMLINUX_MAP > >> > + help > >> > + When modules are built into the kernel, there will be no module name > >> > + associated with its symbols in /proc/kallsyms. Tracers may want to > >> > + identify symbols by module name and symbol name regardless of whether > >> > + the module is configured as loadable or not. > >> > + > >> > + This option generates modules.builtin.ranges in the build tree with > >> > + offset ranges (per ELF section) for the module(s) they belong to. > >> > + It also records an anchor symbol to determine the load address of the > >> > + section. > >> > + > >> > config DEBUG_FORCE_WEAK_PER_CPU > >> > bool "Force weak per-cpu definitions" > >> > depends on DEBUG_KERNEL > >> > diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux > >> > index 5ceecbed31eb..dfb408aa19c6 100644 > >> > --- a/scripts/Makefile.vmlinux > >> > +++ b/scripts/Makefile.vmlinux > >> > @@ -33,6 +33,24 @@ targets += vmlinux > >> > vmlinux: scripts/link-vmlinux.sh vmlinux.o $(KBUILD_LDS) FORCE > >> > +$(call if_changed_dep,link_vmlinux) > >> > > >> > +# module.builtin.ranges > >> > +# --------------------------------------------------------------------------- > >> > +ifdef CONFIG_BUILTIN_MODULE_RANGES > >> > +__default: modules.builtin.ranges > >> > + > >> > +quiet_cmd_modules_builtin_ranges = GEN $@ > >> > + cmd_modules_builtin_ranges = $(real-prereqs) > $@ > >> > + > >> > +targets += modules.builtin.ranges > >> > +modules.builtin.ranges: $(srctree)/scripts/generate_builtin_ranges.awk \ > >> > + modules.builtin vmlinux.map vmlinux.o.map FORCE > >> > + $(call if_changed,modules_builtin_ranges) > >> > + > >> > +vmlinux.map: vmlinux > >> > + @: > >> > + > >> > +endif > >> > + > >> > # Add FORCE to the prerequisites of a target to force it to be always rebuilt. > >> > # --------------------------------------------------------------------------- > >> > > >> > diff --git a/scripts/Makefile.vmlinux_o b/scripts/Makefile.vmlinux_o > >> > index d64070b6b4bc..0b6e2ebf60dc 100644 > >> > --- a/scripts/Makefile.vmlinux_o > >> > +++ b/scripts/Makefile.vmlinux_o > >> > @@ -45,9 +45,12 @@ objtool-args = $(vmlinux-objtool-args-y) --link > >> > # Link of vmlinux.o used for section mismatch analysis > >> > # --------------------------------------------------------------------------- > >> > > >> > +vmlinux-o-ld-args-$(CONFIG_BUILTIN_MODULE_RANGES) += -Map=$@.map > >> > + > >> > quiet_cmd_ld_vmlinux.o = LD $@ > >> > cmd_ld_vmlinux.o = \ > >> > $(LD) ${KBUILD_LDFLAGS} -r -o $@ \ > >> > + $(vmlinux-o-ld-args-y) \ > >> > $(addprefix -T , $(initcalls-lds)) \ > >> > --whole-archive vmlinux.a --no-whole-archive \ > >> > --start-group $(KBUILD_VMLINUX_LIBS) --end-group \ > >> > diff --git a/scripts/generate_builtin_ranges.awk b/scripts/generate_builtin_ranges.awk > >> > new file mode 100755 > >> > index 000000000000..b9ec761b3bef > >> > --- /dev/null > >> > +++ b/scripts/generate_builtin_ranges.awk > >> > @@ -0,0 +1,508 @@ > >> > +#!/usr/bin/gawk -f > >> > >> This forces the gawk to be found always in /usr/bin. For systems where gawk can > >> be located in other places, can we change the Shebang to: > >> > >> diff --git a/scripts/generate_builtin_ranges.awk b/scripts/generate_builtin_ranges.awk > >> index b9ec761b3bef..886251c8d3f7 100755 > >> --- a/scripts/generate_builtin_ranges.awk > >> +++ b/scripts/generate_builtin_ranges.awk > >> @@ -1,4 +1,4 @@ > >> -#!/usr/bin/gawk -f > >> +#!/usr/bin/env gawk -f > >> # SPDX-License-Identifier: GPL-2.0 > >> # generate_builtin_ranges.awk: Generate address range data for builtin modules > >> # Written by Kris Van Hees <kris.van.hees@xxxxxxxxxx> > > > > > > No. We cannot fix it this way. May I ask why if a distro installs gawk somewhere else, the "/usr/bin/env" approach will not work either? I just want to understand that case. > > > > > > I already pointed out this shebang issue. > > > > https://lore.kernel.org/lkml/CAK7LNASLc=ik9QdX4K_XuN=cg+1VcUBk-y5EnQEtOG+qOWaY=Q@xxxxxxxxxxxxxx/ To clarify, I've mentioned this because the patch landed in the linux-next without the fix below. And I did not see there was a build error reporting it. Thanks for sending the link! > > > > > > > > I thought Kris would send a fix up, but > > perhaps people tend to be busy with LPC this week. > > > > > > He did, see https://lore.kernel.org/all/20240912171646.1523528-1-kris.van.hees@xxxxxxxxxx/. Thanks for the link. That worked for me too. > > > > >> Not sure if it's too late? in that case I can send a patch to change this. > > > > > > I can locally fix it up. > > > > Kris agreed with this fix. > > > > > > diff --git a/scripts/Makefile.vmlinux b/scripts/Makefile.vmlinux > > index dfb408aa19c6..1284f05555b9 100644 > > --- a/scripts/Makefile.vmlinux > > +++ b/scripts/Makefile.vmlinux > > @@ -39,7 +39,7 @@ ifdef CONFIG_BUILTIN_MODULE_RANGES > > __default: modules.builtin.ranges > > > > quiet_cmd_modules_builtin_ranges = GEN $@ > > - cmd_modules_builtin_ranges = $(real-prereqs) > $@ > > + cmd_modules_builtin_ranges = gawk -f $(real-prereqs) > $@ > > > > targets += modules.builtin.ranges > > modules.builtin.ranges: $(srctree)/scripts/generate_builtin_ranges.awk \
