On Mon, 2 Mar 2020 at 12:05, Nicolas Mailhot via devel
<devel@xxxxxxxxxxxxxxxxxxxxxxx> wrote:
>
> Le 2020-03-01 02:31, clime a écrit :
> > On Sat, 29 Feb 2020 at 21:50, Nicolas Mailhot via devel
> > <devel@xxxxxxxxxxxxxxxxxxxxxxx> wrote:
> >>
> >> Le samedi 29 février 2020 à 20:30 +0100, clime a écrit :
>
> >> Putting %{dynrel} reconciliation in the rpmbuild -bs stage using
> >> detached file state means that fedpkg local (or checking out git state
> >> and building in mock or copr or OBS or via plain rpmbuild -bs) will
> >> give you the same result as launching fedpkg build.
> >
> > Well, I believe it doesn't. You run:
> >
> > 1) fedpkg local -> produces <somepkg>-1.0-1.fc32 (1 in release because
> > you haven't built that package before)
> > 2) fedpkg build
>
> At that point state is undefined till the build succeeds or not. If the
> build succeeds, the buildsystem will write back a new state. Let’s
> assume it succeeds.
It's undefined if you add more elements to the equation than necessary
(i.e. build system), otherwise it would be well-defined in this case.
>
> > 3) vim <somepkg>.spec and do some change in %description
> > 4) fedpkg commit -m "description improvement"
> > 5) fedpkg local -> produces <somepkg>-1.0-1.fc32
> > 6) fedpkg push -> error because build system pushed meanwhile
>
> Yes, here the packager notices something else has been going on, and he
> needs to merge or rebase. He’d have the same effect if another packager
> had been doing stuff on the package, or a mass rebuild had been going
> on. That’s the distributed decentralized aspect of git, except here the
> packager collided with himself by starting two work threads in parallel
> (one buildsys-side, one local).
The problem is that you launched some process and you need to wait
until it finishes, normally you don't need to anything like that when
working with git. The src.fp.o is a synchronization point of all
developers. Now you are adding there also a build process with which
all the developers need to synchronize. I think it is inconvenient.
>
> > 7) fedpkg pull --rebase (to quickly fix the error)
> > 8) fedpkg build -> builds <somepkg>-1.0-2.fc32
> >
> > To get the correct NVR for the package being built by `fedpkg local`
> > you need to wait for some external process to finish first so that you
> > can pull the new state. Only then you get correct value again.
>
> Sure, that‘s the inconvenient of state: it needs syncing. And the whole
> auto-bump thing intrinsically depends on state not provided manually by
> human beings.
>
> However putting state in git would be no better, because you’d end up
> assuming a build exists before it does, in fact, build (and builds do
> fail because packagers are imperfect humans).
The only assumption is that a certain build will produce certain NEVRA
if it succeeds.
>
> And putting state in buildsys would also not be any better because
> successful production builds end up in changelogs and you want
> changelogs to be tracked.
>
> So, no matter how you re-arange things, you’ll always have a buildsys
> sync-back somewhere (you have it today when humans change their
> changelogs after a failed build; automating that makes the sync-back
> explicit).
Scratch builds can help here a lot.
>
> >> For fedpkg verrel you'll probably want to output a last (saved in
> >> detached file) and next line (probably factorizable by externalizing
> >> the dynrel bump logic in a separate command). That’s more honest
> >> anyway, next may happen or not, when you launch fedpkg verrel, it’s
> >> mere potential (the next commit may bump version and invalidate your
> >> future build).
> >
> > There is the same problem as above with `fedpkg local`, the command
> > won't be giving you correct values at all times.
>
> There is no correct value to give before production build succeeded or
> not. When fedpkg local gives you something today it is lying (just try
> it just before someone mass rebuilds, and then try to commit after this
> mass rebuild: that will fail and you’ll have to merge/rebase).
It gives you something you can rely on if push succeeds.
>
> You’re assuming a simple universe where you’re the only one doing things
> with a package. That’s not a true assumption, many packages have shared
> ownership and maintenance (to say nothing of proven-packagers and releng
> doing things on top of them).
There is only one person doing the actual build for a given release
while multiple persons can participate in development.
>
> > It doesn't need to be the consequence of "automating changelog". It's
> > only a consequence in certain implementations. If the changelog is
> > derived purely from the git state, there aren't these problems
> > anymore.
>
> Sure there are. You're assuming anything committed to git builds before
> actually trying to build it. And expecting manual packager git fixup
> when that’s not the case. Just like in the sync-back state except with
> no tooling enforcement and help.
Who will be fixing the bugs it if not human?
> >> Putting state in detached srpm source files has the following super-
> >> duper property. You can import the srpm or scm checkout between
> >> systems, and they’ll just pick up from the point the previous system
> >> left things, without needing a full scm import.
> >
> > That's nice if there is always just a single provider of the given
> > package in our ecosystem. In case of copr, you can fork a certain repo
> > and continue building there so you will be getting bumped numbers on
> > top of the work done in the original repo. But once the original repo
> > starts producing some builds as well you will be getting that package
> > in two versions in two different coprs and while the original repo
> > might have done less builds than the forked one since the fork
> > happened, it can contain the package with more features because the
> > forked repo was simply playing around.
>
> You will never get official Fedora git releases changed based on the
> status of some copr builds. That takes a deliberate package admin action
> (and that works in the proposal, just fedpkg import the last srpm built
> in copr, that syncs repos).
>
> If you want things to work on the other direction, copr will need to
> read state from Fedora infra one way or another, and it can perfectly
> read state from detached files committed back to Fedora git by koji. And
> then you need a second-level counter for copr, and configure copr for
> using the the second-level counter only. And then if you want OBS to
> build from copr that builds from koji…
>
> I’m not sure multi-level bumping is in scope for the proposal. If it’s
> in scope that makes things more complex (but not a lot more complex).
> Multi-level definitely means the n-th level builder must be able to
> redefine Release structure, and will want the (n-1)th Release exposed as
> variable.
>
> > I can see the advantage of your proposal that the auto-bumping
> > theoretically doesn't need git to work. But it still needs git because
> > you need push the updated %{dynstate} files somewhere. The only way to
> > avoid it would be to build a new srpm together with the rpm where the
> > srpm would be the same as the input srpm except for the updated
> > %{dynstate} files.
>
> That’s why I wrote that the last-step of release bumping belongs in rpm
> proper, at srpm creation stage. Only way I see to get in independent
> from a specific project scm setup. The last build state must be provided
> in srpm data as detached file, and the bumping logic make decisions
> based on this data (we already put state in srpm for reproducible
> builds, IIRC).
>
> > So in case we wouldn't have src.fp.o and the
> > development would be fully decentralized, I could see your proposal to
> > be actually the only way to implement auto-bumping and it would be a
> > nice way. The thing is, the way we are developing today is different
> > from that
>
> If you don’t keep things decentralized you’ll be in a word of pain when
> the scm or buildsys needs to be changed for another implementation (not
> to mention, that’s not a good way to collaborate with other distros).
> That will happen eventually. Web apps are not eternal.
Full decentralization likely means that everyone has its own git repo
and we can only sync by mailing list. I think src.fp.o is a good point
where things can be done and where we can agree on what the packages
that Fedora produce are (meaning we need a canonical source of package
sources, otherwise it would be more complex to put a distribution
together).
Best regards!
clime
>
> Regards,
>
> --
> Nicolas Mailhot
> _______________________________________________
> devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
