On Sat, Jul 23, 2022 at 12:56 PM Neal Gompa <ngompa13@xxxxxxxxx> wrote: > > On Sat, Jul 23, 2022 at 4:14 AM Dan Čermák > <dan.cermak@xxxxxxxxxxxxxxxxxxx> wrote: > > > > Richard Shaw <hobbes1069@xxxxxxxxx> writes: > > > > > Replying in general... > > > > > > I've asked about a "one script to rule them all" a few times over my 10+ > > > year Fedora packaging career and it's fallen on deaf ears. > > > > > > I hope something will happen this time. There should really be only ONE way > > > to determine what packages need to be rebuilt, even if it's not perfect, we > > > can deal with the corner cases but everyone doing their own thing has > > > definitely been worse. > > > > In a perfect world koji or koschei would figure this out themselves and > > perform the rebuilds for us so that we can finally stop thinking about > > build orders and dependencies ourselves. > > The sad part is that Koschei can do it, but the build system folks > have so far refused to enhance Koji and Koschei to do this for > creating *real builds* that are auto-submitted. Alright Neal, this is a bit off-topic, but I'll bite. Auto-submitting real builds is something that I will, except in very narrowly defined exceptions, always disagree with. Until now, automagic builds have only caused trouble. Just look at the mess that's regularly made in the podman/container stack, or by stuff that was automatically submitted by packit. It's one of the reasons why we now have a policy that requires actual people to be responsible for all builds submitted by bots. Even if we had a mechanism to trigger automatic rebuilds of dependent packages (i.e. "I have detected that the sonames of the libraries in this package have changed, let me also rebuild dependent packages for this!") only works *if* (and that's a big *if*) the ABI change isn't accompanied by breaking API changes, as well. What would you want to happen then? I'm pretty sure software isn't smart enough yet to determine in advance if any breaking API changes affect any dependent packages. So how would the mechanism you want work? 1. packager submits a build of libfoo version X that contains an soname bump to koji 2. magic mechanism (TM) detects this ~somehow~ and does ... what? the only thing I can think of is ... 3. magic mechanism (TM) tags the build into an on-demand side-tag N instead of rawhide after it finishes 4. magic mechanism (TM) queries dependent packages 5. magic mechanism (TM) bumps dependent packages with "Rebuilt for libfoo soname bump." 6. magic mechanism (TM) submits builds of dependent packages to on-demand side-tag N 7. magic mechanism (TM) requests bodhi to merge the side tag *if and only if* all builds succeed and are installable That's all well and good so long as everything *just works*, but will instantly break the second any part of this will need human intervention, or if there are any race conditions: - dependent packages need patches for breaking API changes - dependent packages need a compat package libfooXminus1 because they can't be ported - the sets of dependent packages overlap between concurrently created side tags - dependent packages fail to build for unrelated reasons - dependent packages fail to install for related (or unrelated) reasons - packagers have pushed changes to dist-git that they didn't want to get built yet (eugh) - etc. I just see so many points of failure in a process like the one you want that I just can't ever see this becoming a reality unless we enforce much stricter rules around 1) concurrent side tags / using a mutex mechanism for packages in koji, 2) forbidding to push changes to dist-git that must not be built yet, 3) drastically reducing the number of FTBFS / FTI bugs in rawhide ~somehow~. Unless you have ways to address at least some of those issues and failure points with the mechanism that you want (and I cannot see how that would be doable), I don't see automagic rebuilds happening. > We have all the pieces to do it now, especially with being able to > generate random side-tags and merge them freely. > Koschei knows the build order and calculates dependency drift pretty well already. Koschei knows nothing about build order. It regularly submits builds for my packages in the wrong order, or triggers builds at the wrong time, because it orders builds by their "finished" datetime instead of their "actually available" datetime, etc. Almost every time I merge a multi-build side-tag, I need to poke it with a lot of sticks to fix dependency resolution from being stuck for a few packages. > It uses that to trigger scratch builds, we just need it to do real builds > in a generated side tag. But we need a build counter that is > independent of us doing bumps in Git[1]. I can see that differentiating between builds that have the same NVR could be useful (because the build environment or their dependencies were different, etc.), but the proposed "solution" to add another component to the NVR(B) is very disruptive and would probably break all our tools, require adaptations to every component of our build pipeline, etc. But I don't see why you actually *need* independent build counters. At least for packages that use rpmautospec, adding an empty commit with a message like "Rebuilt for the latest version of X" would do it - and that would 1) not require adapting our entire build pipeline and tooling, and 2) actually record the reason for a rebuild in the standard location we have for it (in the dist-git commit log / the package's changelog). > Sadly, that was ripped out of > rpmautospec, making it useless for making package maintenance less > tedious. > > [1]: https://discussion.fedoraproject.org/t/rfc-build-tag-in-rpms/39954 Yes, and I still think that was the right decision. rpmautospec has been astonishingly reliable *because* it's more simple than what you wanted. Adding a - by definition - *fallible* query over a network to something that should never be allowed to fail would've been a terrible idea. Fabio _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
