On 12/04/2024 22:16, Arnaldo Carvalho de Melo wrote: > From: Arnaldo Carvalho de Melo <acme@xxxxxxxxxx> > > $ time tests/reproducible_build.sh vmlinux > Parallel reproducible DWARF Loading/Serial BTF encoding: Ok > > real 1m13.844s > user 3m3.601s > sys 0m9.049s > $ > > If the number of threads started by pahole is different than what was > requests via its -j command line option, it will fail as well as if the > output of 'bpftool btf dump' differs from the BTF encoded totally > serially to one of the detached BTF encoded using reproducible DWARF > loading/BTF encoding. > > Cc: Alan Maguire <alan.maguire@xxxxxxxxxx> > Cc: Kui-Feng Lee <kuifeng@xxxxxx> > Cc: Thomas Weißschuh <linux@xxxxxxxxxxxxxx> > Signed-off-by: Arnaldo Carvalho de Melo <acme@xxxxxxxxxx> > --- > tests/reproducible_build.sh | 56 +++++++++++++++++++++++++++++++++++++ > 1 file changed, 56 insertions(+) > create mode 100755 tests/reproducible_build.sh > great to have a test for this! a few small things below but for the series Reviewed-by: Alan Maguire <alan.maguire@xxxxxxxxxx> Tested-by: Alan Maguire <alan.maguire@xxxxxxxxxx> > diff --git a/tests/reproducible_build.sh b/tests/reproducible_build.sh > new file mode 100755 > index 0000000000000000..9c72d548c2a21136 > --- /dev/null > +++ b/tests/reproducible_build.sh > @@ -0,0 +1,56 @@ > +#!/bin/bash > +# SPDX-License-Identifier: GPL-2.0-only > +# > +# Test if BTF generated serially matches reproducible parallel DWARF loading + serial BTF encoding > +# Arnaldo Carvalho de Melo <acme@xxxxxxxxxx> (C) 2024- > + > +vmlinux=$1 nit: might be worth having a usage check/error for the vmlinux binary here. > +outdir=$(mktemp -d /tmp/reproducible_build.sh.XXXXXX) > + > +echo -n "Parallel reproducible DWARF Loading/Serial BTF encoding: " > + > +test -n "$VERBOSE" && printf "\nserial encoding...\n" > + > +pahole --btf_encode_detached=$outdir/vmlinux.btf.serial $vmlinux suggestion here; what about adding --btf_features=all as this would mean we'd be encoding all the standard kernel features? we'd need to do the same below in the thread loop. without that we're missing out on a few features that the kernel builds use in BTF encoding, and we probably want to ensure that we're testing as close to a real kernel BTF encoding scenario as possible. > +bpftool btf dump file $outdir/vmlinux.btf.serial > $outdir/bpftool.output.vmlinux.btf.serial > + > +nr_proc=$(getconf _NPROCESSORS_ONLN) > + > +for threads in $(seq $nr_proc) ; do > + test -n "$VERBOSE" && echo $threads threads encoding > + pahole -j$threads --reproducible_build --btf_encode_detached=$outdir/vmlinux.btf.parallel.reproducible $vmlinux & > + pahole=$! > + # HACK: Wait a bit for pahole to start its threads > + sleep 0.3s > + # PID part to remove ps output headers > + nr_threads_started=$(ps -L -C pahole | grep -v PID | wc -l) > + > + if [ $threads -gt 1 ] ; then > + ((nr_threads_started -= 1)) > + fi > + > + if [ $threads != $nr_threads_started ] ; then > + echo "ERROR: pahole asked to start $threads encoding threads, started $nr_threads_started" > + exit 1; > + fi > + > + # ps -L -C pahole | grep -v PID | nl > + test -n "$VERBOSE" && echo $nr_threads_started threads started > + wait $pahole > + rm -f $outdir/bpftool.output.vmlinux.btf.parallel.reproducible > + bpftool btf dump file $outdir/vmlinux.btf.parallel.reproducible > $outdir/bpftool.output.vmlinux.btf.parallel.reproducible > + test -n "$VERBOSE" && echo "diff from serial encoding:" > + diff -u $outdir/bpftool.output.vmlinux.btf.serial $outdir/bpftool.output.vmlinux.btf.parallel.reproducible > $outdir/diff > + if [ -s $outdir/diff ] ; then > + echo "ERROR: BTF generated from DWARF in parallel is different from the one generated in serial!" > + exit 1 > + fi > + test -n "$VERBOSE" && echo ----------------------------- > +done > + > +rm $outdir/* > +rmdir $outdir > + > +echo "Ok" > + > +exit 0
