On 06/07/2024 00:55, Dominique Martinet wrote:
> The SOURCE_DATE_EPOCH environment variable is used in reproducible
> builds to set build timestamps and is generally synonymous with
> reproducible builds: instead of making users figure out how to pass
> pahole flags (e.g. explicitly setting PAHOLE_FLAGS in linux) just assume
> that the user wants a reproducible build if this variable is set,
> printing a debug message in verbose mode.
>
> The impact on build time appears to be minimal enough even if we
> incorrectly make this decision (on a Ryzen 7 7735HS):
> $ hyperfine 'sh pahole-test -j --reproducible_build' 'sh pahole-test -j1' 'sh pahole-test -j' -p 'cp .tmp_vmlinux.orig .tmp_vmlinux.btf' -m 4
> Benchmark 1: sh pahole-test -j --reproducible_build
> Time (mean ± σ): 3.991 s ± 0.045 s [User: 7.223 s, System: 3.741 s]
> Range (min … max): 3.950 s … 4.042 s 4 runs
>
> Benchmark 2: sh pahole-test -j1
> Time (mean ± σ): 7.083 s ± 0.095 s [User: 4.805 s, System: 2.242 s]
> Range (min … max): 6.964 s … 7.191 s 4 runs
>
> Benchmark 3: sh pahole-test -j
> Time (mean ± σ): 3.858 s ± 0.089 s [User: 13.447 s, System: 7.078 s]
> Range (min … max): 3.763 s … 3.978 s 4 runs
>
> Summary
> sh pahole-test -j ran
> 1.03 ± 0.03 times faster than sh pahole-test -j --reproducible_build
> 1.84 ± 0.05 times faster than sh pahole-test -j1
>
> Where pahole-test is the pahole command of the linux build (minus -j):
> LLVM_OBJCOPY=objcopy pahole -J --btf_gen_floats --lang_exclude=rust --skip_encoding_btf_inconsistent_proto --btf_gen_optimized .tmp_vmlinux.btf "$@"
>
> And .tmp_vmlinux.orig was the .tmp_vmlinux.btf binary before being
> processed:
> ld -m elf_x86_64 -z noexecstack --no-warn-rwx-segments --emit-relocs --discard-none -z max-page-size=0x200000 --build-id=sha1 --orphan-handling=warn --script=./arch/x86/kernel/vmlinux.lds -o .tmp_vmlinux.btf --whole-archive vmlinux.a .vmlinux.export.o init/version-timestamp.o --no-whole-archive --start-group --end-group
>
> Signed-off-by: Dominique Martinet <asmadeus@xxxxxxxxxxxxx>
>
Reviewed-by: Alan Maguire <alan.maguire@xxxxxxxxxx>
One small thing below, and asuggestion; it'd be good to mention this
behaviour in man-pages/pahole.1 where we discuss the reproducible_build
feature.
Something like the commit message would do, i.e.
"The SOURCE_DATE_EPOCH environment variable is used in reproducible
builds to set build timestamps and is generally synonymous with
reproducible builds so the "reproducible_build" BTF feature is
automatically set when this environment variable is set. This ensures
that kernel builds will generated reproducible BTF in such cases."
Thanks!
Alan
> ---
> v1 -> v2:
> - added message in verbose mode if setting reproducible build through
> this.
> I wasn't quite sure what is least intrusive message-wise -- most
> verbose messages just go to stdout, but using pahole in a script to
> extract some c structs would see this mangled in output which is
> obviously not great. Since --reproducible_build only impacts BTF
> generation at this time, I'm only checking when generating BTF to
> avoid such needless messages -- btf generation always goes through a
> file so there is no risk of mangling the output in this case.
>
yep, that seems like a reasonable approach to me.
> pahole.c | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/pahole.c b/pahole.c
> index c866095c8be6..2792e11265d6 100644
> --- a/pahole.c
> +++ b/pahole.c
> @@ -3705,6 +3705,15 @@ int main(int argc, char *argv[])
> goto out;
> }
>
> + // SOURCE_DATE_EPOCH being set means whoever called us likeely tries
nit: typo on likely. also (tho it's not always 100% consistent I admit
I'd use a multi-line comment style
/*
* SOURCE_DATE_EPOCH being set means whoever called us likely tries
...
*/
> + // to do a reproducible build. The setting only impacts btf_encode code
> + // so only check in this case to avoid needless messages.
> + if (btf_encode && getenv("SOURCE_DATE_EPOCH")) {
> + if (!conf_load.reproducible_build && global_verbose)
> + printf("pahole: auto-enabling reproducible build (SOURCE_DATE_EPOCH set)\n");
> + conf_load.reproducible_build = true;
> + }
> +
> if (languages.str && parse_languages())
> return rc;
>
