On Fri, Mar 08, 2024 at 09:07:30AM +0100, Petr Pisar wrote: > V Thu, Mar 07, 2024 at 12:39:37PM +0000, Zbigniew Jędrzejewski-Szmek napsal(a): > > The effort to make package builds in Fedora reproducible has picked up steam again. > > We now have a new website: https://docs.fedoraproject.org/en-US/reproducible-builds > > and an issue tracker: https://pagure.io/fedora-reproducible-builds/project > > and a matrix room: https://matrix.to/#/#reproducible-builds:fedora.im > > > > We've done a mini rebuild using [1] for the package list [2] and results are at [3]. > > (The result is a json dump of rpmdiff output by package. Generally, "" means > > the rebuild was identical except for variable metadata, and non-empty > > output else means that the rebuild was different.) > > > > [1] https://github.com/keszybz/fedora-repro-build > > [2] https://fedorapeople.org/~zbyszek/builds-2024-02-fc41-filtered.txt > > [3] https://fedorapeople.org/~zbyszek/builds-2024-02-fc41-filtered.results.txt > > > Is this mini rebuild a one-shot thing, or are are you going to rebuild the > packages repeatedly or use the results for something significant? I ask > because I spotted some discrepancies in those text files: Both ;) I'm working on the tooling to do the rebuilds, and the code is still a bit rough, but also I'm still figuring out the design. Once that done, we hope to hook this up to rebuilderd to get a continous rebuild with a nice dashboard (https://wiki.archlinux.org/title/Rebuilderd). For now, the results are being used to figure out a list of things to fix. > (1) Some packages are listed twice, with different NEVRAs. E.g. > perl-Alien-pkgconf or perl-RDF-RDFa-Generator. This is because they were built multiple times in koji. > (2) Both perl-Alien-pkgconf NEVRAs reports a differing > /usr/lib64/perl5/vendor_perl/auto/share/dist/Alien-pkgconf/status.json > content. That content looks likes this: > > {"libs":"-lpkgconf","version":"2.1.0","install_type":"system","cflags":"-I/usr/include/pkgconf","dll":"/lib64/libpkgconf.so.4.0.0"} > > That means you had to perform rebuilds of the same NEVRA with different > libpkgconf-devel packages in the build roots. That looks like a bug in your > mini rebuild scheduler. Diffoscope says: ├── ./usr/lib64/perl5/vendor_perl/auto/share/dist/Alien-pkgconf/status.json │ ├── Pretty-printed │ │┄ Ordering differences only │ │ @@ -1,7 +1,7 @@ │ │ { │ │ "libs": "-lpkgconf", │ │ + "dll": "/lib64/libpkgconf.so.4.0.0", │ │ "version": "2.1.0", │ │ - "install_type": "system", │ │ "cflags": "-I/usr/include/pkgconf", │ │ - "dll": "/lib64/libpkgconf.so.4.0.0" │ │ + "install_type": "system" │ │ } It would be great to sort the dictionary to avoid this randomness. > (3) Some packages listed in builds-2024-02-fc41-filtered.txt are missing from > builds-2024-02-fc41-filtered.results.txt. E.g. > perl-CPAN-Plugin-Sysdeps-0.73-1.fc41 is listed as COMPLETE, yet results are > missing. Yes. Some packages failed to build, and then I finished the build early because there were already enough interesting results. (The few failures I looked at were caused by differences in BR between architectures. This is currently a corner case that I'm not sure how to deal with. Most of the time, using a srpm from a different architecture works fine, but in some cases the set of installed packages would differ, and then I can't figure out which version of the rpm for the local architecture would have been used and buildroot creation fails. I would be happy to describe the problem in more detail. It's also possible that other packages FTBFS, I didn't look into this and I didn't save the logs.) > (4) dnf5-5.1.13-1.fc41.src reports changes in Requires (e.g. "removed > REQUIRES createrepo_c"). That again looks like you built the same NEVRA in > different build roots (for some reason "%bcond_without tests" flipped). Yes, the build root is different. I install the package set that was used for the main package build and just call mock with that and it does both srpm and the binary rpms there. But koji does the srpm build in a separate buildroot that is smaller. It would be fairly easy to get the buildroot listing for the srpm and build the srpm separately. This wouldn't actually doesn't solve the problem fully, because koji will use a random build architecture, so we may potentially hit the problem that was described above for noarch builds. For now, I'm taking the pragmatic approach of ignoring changes in PROVIDES/REQUIRES for the srpm. We know that those are not important. OTOH, for example changes in the source files are unexpected. I filed https://src.fedoraproject.org/rpms/stratisd/pull-request/6 for stratisd to stop rewriting the source tarballs (it was done irreproducibly, but I think it's better to not do this at all). > All that means you might hunting ghosts instead of real bugs. The rebuild process is surprisingly reliable. Per the discussion above, we have some mix of real issues and (unexpected) randomness. Of course such randomness usually is not a bug, but I hope we can agree to get rid of any such cases to make the builds reproducible. Zbyszek -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
