On Tue, 3 Mar 2020 at 09:22, Nicolas Mailhot
<nicolas.mailhot@xxxxxxxxxxx> wrote:
>
> Le 2020-03-02 14:45, clime a écrit :
> > On Mon, 2 Mar 2020 at 12:05, Nicolas Mailhot via devel
> > <devel@xxxxxxxxxxxxxxxxxxxxxxx> wrote:
> >>
> >> Le 2020-03-01 02:31, clime a écrit :
> >> > On Sat, 29 Feb 2020 at 21:50, Nicolas Mailhot via devel
> >> > <devel@xxxxxxxxxxxxxxxxxxxxxxx> wrote:
> >> >>
> >> >> Le samedi 29 février 2020 à 20:30 +0100, clime a écrit :
> >>
> >> >> Putting %{dynrel} reconciliation in the rpmbuild -bs stage using
> >> >> detached file state means that fedpkg local (or checking out git state
> >> >> and building in mock or copr or OBS or via plain rpmbuild -bs) will
> >> >> give you the same result as launching fedpkg build.
> >> >
> >> > Well, I believe it doesn't. You run:
> >> >
> >> > 1) fedpkg local -> produces <somepkg>-1.0-1.fc32 (1 in release because
> >> > you haven't built that package before)
> >> > 2) fedpkg build
> >>
> >> At that point state is undefined till the build succeeds or not. If
> >> the
> >> build succeeds, the buildsystem will write back a new state. Let’s
> >> assume it succeeds.
> >
> > It's undefined if you add more elements to the equation than necessary
> > (i.e. build system), otherwise it would be well-defined in this case.
> >
> >>
> >> > 3) vim <somepkg>.spec and do some change in %description
> >> > 4) fedpkg commit -m "description improvement"
> >> > 5) fedpkg local -> produces <somepkg>-1.0-1.fc32
> >> > 6) fedpkg push -> error because build system pushed meanwhile
> >>
> >> Yes, here the packager notices something else has been going on, and
> >> he
> >> needs to merge or rebase. He’d have the same effect if another
> >> packager
> >> had been doing stuff on the package, or a mass rebuild had been going
> >> on. That’s the distributed decentralized aspect of git, except here
> >> the
> >> packager collided with himself by starting two work threads in
> >> parallel
> >> (one buildsys-side, one local).
> >
> > The problem is that you launched some process and you need to wait
> > until it finishes, normally you don't need to anything like that when
> > working with git.
>
> But fedpkg build is a release to production process. It’s not a dev
> staging process. In release processes, actually doing the release is not
> an inconvenient optional check.
Well, you could have a situation when somebody wants to immediately
continue working after making a release. Probably rare but could
happen and if build time of a package is long (libreOffice/firefox),
this could hit you as an inconvenience. Or do you want to push-back
immediately after srpm build when you don't yet know whether the build
will succeed? That wasn't clear to me before.
>
> You're asked to autobump a “Release” field. You’re asked to automate a
> changelog which is effectively a release changelog, not dev changelog
> (the dev changelog lives in git, people want a good changelog in rpm
> proper to track release not git history).
I am not suggesting to use raw git commit messages for the changelog
but instead content of annotated tags which can be initially
prepopulated by commit messages but can be then edited to any extent
and even the way the edit window for annotated tag is prepopulated
might be configurable (to .e.g. load content from a file in which case
you would probably skip the editing part). And I see the annotated
tags as the actual releases.
>From https://git-scm.com/docs/git-tag:
Annotated tags are meant for release while lightweight tags are meant
for private or temporary object labels.
They were designed to represent releases and because we are using git
I don't see why we should avoid using annotated tags for releases.
On src.fp.o, there is also already support for this: e.g.
https://src.fedoraproject.org/rpms/2ping/releases
The release needs building (koji), and distribution (bodhi + mirrors)
but that's stuff that needs to be done with the release. Scratch
builds can be used to make sure that the package builds and in the
worst case, a new release needs to be done that accounts for changed
build environment. But I don't think we need to throw the whole
concept of annotated tags = releases because of it.
Of course, everything here can be made opt-in. As an approach somebody
likes and somebody doesn't.
I think this is where our approaches really differ, you see release as
something that happens somewhere else than in git, but I see the
release as something that happens in git. And afterwards we need to
get that released content to users.
I think we will never be able to reconciliate our views because they
are different from the very base. In the end, what matters is
implementation complexity and ease of use. I can present to you the
things that need to be done for this to work in a detailed way
(although it would take a bit of time to put into a document) but I
also know it is a doable and it is actually fairly simple when not
accounting for time spending on pull requests and getting them
accepted. Your approach seems to be intuitively difficult to
implement, either locking implementation (i am sure there would be
quite unpleasant cases when the repo just stays locked even if it
should be already unlocked, at least initially) or the fail after repo
change - koji would need to listen for changes in dist-git and then
abort builds based on that, which is doable but then you need
authentication between the two systems (i.e koji needs to have write
access to dist-git which is probably not the case today because the
releng script to do mass rebuilds works separately and only invokes
builds in koji) and you need some job that gets parsed changelog and
NVR and does the pushing back to dist-git, which needs to be strictly
separated from srpm/rpm builds due to security. Again, quite fairly
complex to introduce something like that into a system like koji.
Maybe you would like to bypass koji and create some external service
that listens for dist-git and koji events and it is doing what you
describe but introduction of this service is again quite a big task
especially if one needs access to write to dist-git. And i think I
would be able to find out more smaller or bigger obstacles when
actually really trying to evaluate how difficult this would be.
The thing is that the I am not that much of a fan of "push into
dist-git can make build fail" - it seems to me like something nobody
expects so I don't want to be spending time figuring this out so that
I can give you arguments that would be acceptable for you. If,
however, you will spend time on "what piece of code need to be
inserted where" in the infrastructure, I'll spend time on it as well
for what I am proposing and we can evaluate what would be easier.
I am basically suggesting a small preprocessing step before sprm
build. Cloning dist-git repo is already somewhere there in Koji, there
just needs to be one more command invocation before everything
continues the same way as before. No write access do dist-git needed
which is already a big relief.
>
> The *single* *only* reason good release numbers and good changelog
> matter is because the result will be released to third parties. koshei,
> or scratch builds, or whatever, do not care about those.
I agree they matter.
>
> A normal git flow would lock the branch at release time to let the
> release manager do his releasing (merge window closed in Linus’s terms).
> That’s option 1 I gave you.
>
> Alternatively, you can use option 2, the “I feel lucky” approach, let
> everything open, and abort the release if merge-back failed. A packager
> that changes his package before the production build is finished will
> probably want it canceled anyway (what is the point of risking breaking
> builds of other packages by pushing an unfinished package to the shared
> koji buildroot)?
>
> Either option does not stop the packager to do anything he wants in his
> local branches. They just require him to merge/rebase when syncing with
> the centralized master Fedora state. That’s a core element of git
> architecture, be it in Fedora or elsewhere.
>
> You can not avoid a buildsys merge-back when doing production builds.
Merge backs by build system can be avoided, however. Why do you think
they can't be avoided?
There is the case of mass rebuilds but this is pretty much a one-shot
event for each branch and it's not a merge-back we are talking about
here....I would need to see the implementation today to be sure but in
F12, it used be a separate script doing this:
https://fedoraproject.org/wiki/Fedora_12_Mass_Rebuild
> The merge-back can be explicit and automatic (as I proposed), or you can
> rely on informal meatware for it, but it will exist. Since the whole
> point of the request is to lower dependence on meatware, why on hell
> would you want to keep meatware as part of the whole process?
To me the whole point of the request is "make packager's life easier".
If it goes hand-to-hand with automation (which it does by definition),
then even better.
>
> The only “cost” for the packager is to wait for his build to finish
> before continuing to change the package (or merge/rebase). That’s not an
> horrific cost. No one forced the packager to start a build before he was
> ready. No one forced the packager to perform a production build instead
> of a scratch build if he just wanted to check partial changes. If shit
> happens, and he realizes before the build end something is not ok,
> aborting the build on failed merge back is helping the packager.
>
> Or am I missing something? “normally you don't need to anything like
> that when working with git” is not a clear technical point.
It's rather a point about what people are used to and what should be
taken into account as well. If we want more projects on src.fp.o
(wouldn't be bad, right?), then we shouldn't introduce an unexpected
complexity that people would need to learn about in probably not that
pleasant manner (unless they read dist-git documentation or there is
some banner somewhere saying "don't push while building").
Best regards!
clime
>
> Regards,
>
> --
> Nicolas Mailhot
> _______________________________________________
> packaging mailing list -- packaging@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to packaging-leave@xxxxxxxxxxxxxxxxxxxxxxx
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/packaging@xxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
packaging mailing list -- packaging@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to packaging-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/packaging@xxxxxxxxxxxxxxxxxxxxxxx
