Re: repoquery-fu (was Re: Retiring the pcre package from Fedora)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sat, Jul 23, 2022 at 12:56 PM Neal Gompa <ngompa13@xxxxxxxxx> wrote:
>
> On Sat, Jul 23, 2022 at 4:14 AM Dan Čermák
> <dan.cermak@xxxxxxxxxxxxxxxxxxx> wrote:
> >
> > Richard Shaw <hobbes1069@xxxxxxxxx> writes:
> >
> > > Replying in general...
> > >
> > > I've asked about a "one script to rule them all" a few times over my 10+
> > > year Fedora packaging career and it's fallen on deaf ears.
> > >
> > > I hope something will happen this time. There should really be only ONE way
> > > to determine what packages need to be rebuilt, even if it's not perfect, we
> > > can deal with the corner cases but everyone doing their own thing has
> > > definitely been worse.
> >
> > In a perfect world koji or koschei would figure this out themselves and
> > perform the rebuilds for us so that we can finally stop thinking about
> > build orders and dependencies ourselves.
>
> The sad part is that Koschei can do it, but the build system folks
> have so far refused to enhance Koji and Koschei to do this for
> creating *real builds* that are auto-submitted.

Alright Neal, this is a bit off-topic, but I'll bite.

Auto-submitting real builds is something that I will, except in very
narrowly defined exceptions, always disagree with.
Until now, automagic builds have only caused trouble. Just look at the
mess that's regularly made in the podman/container stack, or by stuff
that was automatically submitted by packit. It's one of the reasons
why we now have a policy that requires actual people to be responsible
for all builds submitted by bots.

Even if we had a mechanism to trigger automatic rebuilds of dependent
packages (i.e. "I have detected that the sonames of the libraries in
this package have changed, let me also rebuild dependent packages for
this!") only works *if* (and that's a big *if*) the ABI change isn't
accompanied by breaking API changes, as well. What would you want to
happen then? I'm pretty sure software isn't smart enough yet to
determine in advance if any breaking API changes affect any dependent
packages.

So how would the mechanism you want work?

1. packager submits a build of libfoo version X that contains an
soname bump to koji
2. magic mechanism (TM) detects this ~somehow~ and does ... what? the
only thing I can think of is ...
3. magic mechanism (TM) tags the build into an on-demand side-tag N
instead of rawhide after it finishes
4. magic mechanism (TM) queries dependent packages
5. magic mechanism (TM) bumps dependent packages with "Rebuilt for
libfoo soname bump."
6. magic mechanism (TM) submits builds of dependent packages to
on-demand side-tag N
7. magic mechanism (TM) requests bodhi to merge the side tag *if and
only if* all builds succeed and are installable

That's all well and good so long as everything *just works*, but will
instantly break the second any part of this will need human
intervention, or if there are any race conditions:

- dependent packages need patches for breaking API changes
- dependent packages need a compat package libfooXminus1 because they
can't be ported
- the sets of dependent packages overlap between concurrently created side tags
- dependent packages fail to build for unrelated reasons
- dependent packages fail to install for related (or unrelated) reasons
- packagers have pushed changes to dist-git that they didn't want to
get built yet (eugh)
- etc.

I just see so many points of failure in a process like the one you
want that I just can't ever see this becoming a reality unless we
enforce much stricter rules around 1) concurrent side tags / using a
mutex mechanism for packages in koji, 2) forbidding to push changes to
dist-git that must not be built yet, 3) drastically reducing the
number of FTBFS / FTI bugs in rawhide ~somehow~.

Unless you have ways to address at least some of those issues and
failure points with the mechanism that you want (and I cannot see how
that would be doable), I don't see automagic rebuilds happening.

> We have all the pieces to do it now, especially with being able to
> generate random side-tags and merge them freely.

> Koschei knows the build order and calculates dependency drift pretty well already.

Koschei knows nothing about build order.
It regularly submits builds for my packages in the wrong order, or
triggers builds at the wrong time, because it orders builds by their
"finished" datetime instead of their "actually available" datetime,
etc. Almost every time I merge a multi-build side-tag, I need to poke
it with a lot of sticks to fix dependency resolution from being stuck
for a few packages.

> It uses that to trigger scratch builds, we just need it to do real builds
> in a generated side tag. But we need a build counter that is
> independent of us doing bumps in Git[1].

I can see that differentiating between builds that have the same NVR
could be useful (because the build environment or their dependencies
were different, etc.), but the proposed "solution" to add another
component to the NVR(B) is very disruptive and would probably break
all our tools, require adaptations to every component of our build
pipeline, etc.

But I don't see why you actually *need* independent build counters. At
least for packages that use rpmautospec, adding an empty commit with a
message like "Rebuilt for the latest version of X" would do it - and
that would 1) not require adapting our entire build pipeline and
tooling, and 2) actually record the reason for a rebuild in the
standard location we have for it (in the dist-git commit log / the
package's changelog).

> Sadly, that was ripped out of
> rpmautospec, making it useless for making package maintenance less
> tedious.
>
> [1]: https://discussion.fedoraproject.org/t/rfc-build-tag-in-rpms/39954

Yes, and I still think that was the right decision.
rpmautospec has been astonishingly reliable *because* it's more simple
than what you wanted.
Adding a - by definition - *fallible* query over a network to
something that should never be allowed to fail would've been a
terrible idea.

Fabio
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux