Hi Andrii,
On 2024-12-12 11:23:03-0800, Andrii Nakryiko wrote:
> On Tue, Dec 10, 2024 at 10:24 PM Thomas Weißschuh <linux@xxxxxxxxxxxxxx> wrote:
> > On 2024-12-11 00:17:02+0000, Ihor Solodrai wrote:
> > > On Tuesday, December 10th, 2024 at 3:23 PM, Thomas Weißschuh <linux@xxxxxxxxxxxxxx> wrote:
> > >
> > > >
> > > >
> > > > Pahole v1.27 added a new BTF generation feature to support
> > > > reproducibility in the face of multithreading.
> > > > Enable it if supported and reproducible builds are requested.
> > > >
> > > > As unknown --btf_features are ignored, avoid the test for the pahole
> > > > version to keep the line readable.
> > > >
> > > > Fixes: b4f72786429c ("scripts/pahole-flags.sh: Parse DWARF and generate BTF with multithreading.")
> > > > Fixes: 72d091846de9 ("kbuild: avoid too many execution of scripts/pahole-flags.sh")
> > > > Link: https://lore.kernel.org/lkml/4154d202-5c72-493e-bf3f-bce882a296c6@xxxxxxxxxx/
> > > > Link: https://lore.kernel.org/lkml/20240322-pahole-reprodicible-v1-1-3eaafb1842da@xxxxxxxxxxxxxx/
> > > > Signed-off-by: Thomas Weißschuh linux@xxxxxxxxxxxxxx
> > > >
> > > > ---
> > > > scripts/Makefile.btf | 1 +
> > > > 1 file changed, 1 insertion(+)
> > > >
> > > > diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf
> > > > index c3cbeb13de503555adcf00029a0b328e74381f13..da23265bc8b3cf43c0a1c89fbc4f53815a290e13 100644
> > > > --- a/scripts/Makefile.btf
> > > > +++ b/scripts/Makefile.btf
> > > > @@ -22,6 +22,7 @@ else
> > > >
> > > > # Switch to using --btf_features for v1.26 and later.
> > > > pahole-flags-$(call test-ge, $(pahole-ver), 126) = -j$(JOBS) --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func,decl_tag_kfuncs
> > > > +pahole-flags-$(if $(KBUILD_BUILD_TIMESTAMP),y) += --btf_features=reproducible_build
> > >
> > > Hi Thomas,
> > >
> > > There are a couple of issues with reproducible_build flag which I
> > > think are worth mentioning here. I don't know all the reasons behind
> > > adding this now, and it's optional too, so feel free to discard my
> > > comments.
> > >
> > > Currently with this flag, the BTF output is deterministic for a given
> > > order of DWARF compilation units. So the BTF will be the same for the
> > > same vmlinux binary. However, if the vmlinux is rebuilt due to an
> > > incremental change in a source code, my understanding is that there is
> > > no guarantee that DWARF CUs will be in the same order in the binary.
> >
> > The goal behind reproducible builds is to produce bit-by-bit idential
> > binaries. If the CUs are in a different order then that requirement
> > would have been broken there already.
>
> I'm curious, how do we guarantee that we get bit-by-bit identical
> DWARF? Do we enforce the order of linking of .o files into the final
> vmlinux? Is this described anywhere?
The CU order has to be fixed, otherwise the non-debugging parts of the
binary would not be reproducible either.
For docs is Documentation/kbuild/reproducible-builds.rst, the linked
reproducible-builds.org project has much more information.
Also besides reproducible builds, lots of kernel components rely
(accidentally or intentionally) on a stable initialization order, which
is also defined by linking order.
