Re: Why does Koschei not run real builds?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Thu, 16 Apr 2020 at 09:51, Pierre-Yves Chibon <pingou@xxxxxxxxxxxx> wrote:
>
> On Thu, Apr 16, 2020 at 01:57:40AM +0200, clime wrote:
> > On Thu, 16 Apr 2020 at 00:52, Dan Čermák <dan.cermak@xxxxxxxxxxxxxxxxxxx> wrote:
> > >
> > > clime <clime@xxxxxxxxxxxxxxxxx> writes:
> > >
> > > > On Mon, 13 Apr 2020 at 10:55, Dan Čermák <dan.cermak@xxxxxxxxxxxxxxxxxxx> wrote:
> > > >>
> > > >> Hi list,
> > > >>
> > > >> my question is pretty much $subject: Why doesn't Koschei kick of
> > > >> real builds off packages on dependency changes? From my naive POV that
> > > >> looks like the missing piece to give us the "OBS-experience". Having
> > > >> that at least in Rawhide sounds like a good thing to me.
> > > >
> > > > Dan, can I have some basic questions to this because I don't know OBS.
> > > >
> > > > Could you describe the feature in more detail with regards to
> > > > auto-rebuilding and when it is useful?
> > >
> > > In a nutshell: OBS will in its default mode rebuild each package once
> > > one of its direct or indirect dependencies changes.
> > >
> > > That is pretty useful, because as a maintainer you can just update a
> > > library and you don't have to do a thing to get dependent packages
> > > rebuilt. So no more "unannounced SONAME bump", "please rebuild XYZ" and
> > > "need a provenpackager to rebuild dependent packages of ABC" emails on
> > > devel. Also, if a package fails to build due to an update, it will be
> > > noticed right away and not until the next mass rebuild.
> > >
> > > Additionally updating a bunch of packages will no longer require that
> > > you figure out the build order yourself: the build system figures it out
> > > itself by rebuilding your packages until the transitive dependencies
> > > stop changing.
> > >
> > > All of this is of course only really viable for Rawhide and already
> > > released Fedora branches should not be run like this, because one wrong
> > > update could wreck the whole distro.
> >
> > Thanks, that was a nice explanation. I personally believe there is a
> > good solution in extending koji and rpm.
> >
> > koji to be able to rebuild the same source package again and again
> > while passing a different increasing number to rpmbuild through
> > --define and rpm to put that number into rpm name if it was specified
> > to the following position:
> >
> > python3-colcon-ros-bundle-0.0.14-1.fc31.<buildid>.noarch.rpm
> >
> > It can be just a short build id specific for the given package (i.e.
> > how many builds there were for the given package). There might be a
> > little bit of trouble to keep <buildid> meaningfully increasing in
> > case of multiple parallel builds but I think it should be possible.
> >
> > An advantage over the rpmautospec approach is that this will just work
> > for all the packages out of the box, i.e. no matter what macros they
> > are using.

Pierre, maybe we can start understanding each other here.

>
> Including the build number in the release field was part of the ideas submitted
> for feedback when we started looking at the auto-release question and there
> seemed to be a consensus about not wanting to have the release depend on the
> build system.

I have a different understanding here :). What you did in rpmautospec
means that release _is_ dependant on build system because it is
dependant on information owned by it.

You can have two slightly different ways to implement the dependency,
one is externally using public api so that build system doesn't know
what is happening. The second is to incorporate build system into the
solution and let it generate the field or its part.

But in both these implementations, the release field is dependent on
build system because if there is no build system, there is no valid
release and because you rely on how certain switches (electrons) move
in build system's db.

Imho, people asked you to implement a solution where the information
for Release field is drawn purely from git (dist-git), i.e.

https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx/message/QHZHOOXGR6REIV4HK4BKKNNWVIY6POYE/

to cite the first response.

That's when dist-git becomes "single source of truth" for the release
field. It is also how things are today.

...So I thought you just eventually ignored the people and ignored the
consensus because you eventually pulled the build information in.

>
> rpmautospec allows to rebuild a commit multiple times and you will get a
> different release everytime, that release is computed using the information from
> the build history of the package, made available via git tags, so the build
> system is not involved in the computing of the release field (which thus can be
> reproduced locally).

Build system _is_ involved in computing the release field because you
are querying its API.

And therefore the results _cannot_ be reproduced locally because,
imho, a solution where you need to query some central service through
network does not have "locally-reproducible" property. Or if you avoid
that by returning always 1 as a result, that also means the solution
does not have "local-reproducibility" property because locally, you
are just always returning a fake constant instead of the real value.

(The dance you do around chroot with tags is only an implementation
detail - it doesn't change essential properties of the solution.)

Basically, what you did with rpmautospec is that:
- you provided a workaround to the koji NVR uniqueness constraint so
that auto-rebuilds can work (but only for those packages actually
using rpmautospec, not for all packages)
- you provided a solution for automatic release and changelog generation

But these are essentially two separate problems and it is better to
treat them as such. You basically tried to do two things at once.
That's not a good approach imho because you will not be able to solve
both things to a satisfactory level.

----

As for my proposal, I suggested to use a different field than Release,
which means values for that field (BUILD_ID) wouldn't be locally
reproducible. But here it shouldn't matter because that BUILD_ID will
be something to enable auto-rebuilds but not something packager
otherwise needs to care about.

There is a clear separation of concerns
(https://en.wikipedia.org/wiki/Separation_of_concerns). dist-git data
produce Release in a predictable manner. Build system produces
increasing build ids for rpm builds and it doesn't care about anything
else. It's very simple even though there will be some work to actually
implement it.

clime

>
>
> Pierre
> _______________________________________________
> devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux