On Tue, Dec 10, 2024 at 10:24 PM Thomas Weißschuh <linux@xxxxxxxxxxxxxx> wrote:
>
> Hi Ihor,
>
> On 2024-12-11 00:17:02+0000, Ihor Solodrai wrote:
> > On Tuesday, December 10th, 2024 at 3:23 PM, Thomas Weißschuh <linux@xxxxxxxxxxxxxx> wrote:
> >
> > >
> > >
> > > Pahole v1.27 added a new BTF generation feature to support
> > > reproducibility in the face of multithreading.
> > > Enable it if supported and reproducible builds are requested.
> > >
> > > As unknown --btf_features are ignored, avoid the test for the pahole
> > > version to keep the line readable.
> > >
> > > Fixes: b4f72786429c ("scripts/pahole-flags.sh: Parse DWARF and generate BTF with multithreading.")
> > > Fixes: 72d091846de9 ("kbuild: avoid too many execution of scripts/pahole-flags.sh")
> > > Link: https://lore.kernel.org/lkml/4154d202-5c72-493e-bf3f-bce882a296c6@xxxxxxxxxx/
> > > Link: https://lore.kernel.org/lkml/20240322-pahole-reprodicible-v1-1-3eaafb1842da@xxxxxxxxxxxxxx/
> > > Signed-off-by: Thomas Weißschuh linux@xxxxxxxxxxxxxx
> > >
> > > ---
> > > scripts/Makefile.btf | 1 +
> > > 1 file changed, 1 insertion(+)
> > >
> > > diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf
> > > index c3cbeb13de503555adcf00029a0b328e74381f13..da23265bc8b3cf43c0a1c89fbc4f53815a290e13 100644
> > > --- a/scripts/Makefile.btf
> > > +++ b/scripts/Makefile.btf
> > > @@ -22,6 +22,7 @@ else
> > >
> > > # Switch to using --btf_features for v1.26 and later.
> > > pahole-flags-$(call test-ge, $(pahole-ver), 126) = -j$(JOBS) --btf_features=encode_force,var,float,enum64,decl_tag,type_tag,optimized_func,consistent_func,decl_tag_kfuncs
> > > +pahole-flags-$(if $(KBUILD_BUILD_TIMESTAMP),y) += --btf_features=reproducible_build
> >
> > Hi Thomas,
> >
> > There are a couple of issues with reproducible_build flag which I
> > think are worth mentioning here. I don't know all the reasons behind
> > adding this now, and it's optional too, so feel free to discard my
> > comments.
> >
> > Currently with this flag, the BTF output is deterministic for a given
> > order of DWARF compilation units. So the BTF will be the same for the
> > same vmlinux binary. However, if the vmlinux is rebuilt due to an
> > incremental change in a source code, my understanding is that there is
> > no guarantee that DWARF CUs will be in the same order in the binary.
>
> The goal behind reproducible builds is to produce bit-by-bit idential
> binaries. If the CUs are in a different order then that requirement
> would have been broken there already.
I'm curious, how do we guarantee that we get bit-by-bit identical
DWARF? Do we enforce the order of linking of .o files into the final
vmlinux? Is this described anywhere?
>
> For an incremental build a full relink with *all* CUs is done, not only
> the changed once, so the order should always be the same.
The concern here is whether linker guarantees that CUs' DWARF data
will be appended in exactly the same order in such case?
>
> > At the same time, reproducible_build slows down BTF generation by
> > 30-50%, maybe more depending on the kernel config.
>
> If a user explicitly requests reproducibility then they should get it,
> even if it is slower.
>
> > Hopefully these problems will be solved in upcoming pahole releases.
>
> I don't see it as big problem. This is used for release builds, not
> during development.
>
> > Question: why KBUILD_BUILD_TIMESTAMP flag? Isn't it more appropriate
> > to use a separate flag for this particular feature?
>
> Adding an additional variable would need to be documented and would
> makes the feature harder to use. KBUILD_BUILD_TIMESTAMP already needs to
> be set by the user if they are building for reproducibility.
>
> > > ifneq ($(KBUILD_EXTMOD),)
> > > module-pahole-flags-$(call test-ge, $(pahole-ver), 126) += --btf_features=distilled_base
> > >
> > > ---
> > > base-commit: 7cb1b466315004af98f6ba6c2546bb713ca3c237
> > > change-id: 20241124-pahole-reproducible-2b879ac8bdab
> > >
> > > Best regards,
> > > --
> > > Thomas Weißschuh linux@xxxxxxxxxxxxxx
> > >
> > >
> >
