On Sat, 29 Feb 2020 at 16:12, Nicolas Mailhot via devel
<devel@xxxxxxxxxxxxxxxxxxxxxxx> wrote:
>
> Le samedi 29 février 2020 à 13:49 +0100, clime a écrit :
> > On Sat, 29 Feb 2020 at 10:15, Nicolas Mailhot via devel
> > <devel@xxxxxxxxxxxxxxxxxxxxxxx> wrote:
> > > Hi,
> > >
> > > Anyway, here is what I would personnaly consider a robust,
> > > inclusive,
> > > and future-proof design:
> >
> > I will need to ask some questions to really understand it.
> >
> > > — a %{use_dynstate} rpm variable enables dynamic changelog/release
> > > behaviour
> > > — probably initialy set to false distro-wide, to let volunteers
> > > test
> > > the thing by setting it to true iin their specs,
> > > — then to true (opt-out), when kinks have been fixed, and
> > > FPC/FESCO
> > > greenlights general availability
> > >
> > > – when activated, last changelog, evr and %{dynrel} state are saved
> > > in one or several detached files
> >
> > You mean the last changelog, evr, and %{dynrel} are stored once
> > %{use_dynstate} is set and and after one invokes fedpkg commit?
> > I think there should be some explicit action (e.g. the commit) to
> > generate the files after I set the %{use_dynstate} value to true in
> > the spec file.
>
> Once a spec sets %{use_dynstate} to true changelog, last computed
> changelog, ev, neutralized-r, and %{dynrel} are taken from detached
> files. neutralized-r is the evaluation of Release with %{dist} set to a
> neutral value (for example “-”).
>
> State is stored in files included in the srpm to be able to import to
> and from Fedora git (pretty much a hard requirement if tooling is to be
> kept Fedora-git neutral, which is itself a hard requirement to be able
> to interact with packaging work existing outside Fedora git).
>
> You need the last computed ev and neutralized-r to decide whether
> %{dynrel} can be kept, needs bumping, or should be reset to 1.
>
> I don’t trust mixed mode, KISS, changelog is detached or not, don’t try
> to have it both ways, here lies madness and confusion.
>
> > How is %{dynrel} computed here at this stage
>
> Nothing is computed at this stage.
>
> Detached changelog state, is a commit property (except for build date).
> It is computed at fedpkg commit time:
> – if detached changelog data is absent
> → move in-spec changelog data to the detached file if data exists,
> → otherwise init detached data to nothing
> – after the first fedpkg commit with %{use_dynstate} set to true, the
> detached changelog file exists and the in-spec changelog has been
> removed.
> – others have detailed possible fedpkg commit strategies to stuff
> changelog with new info
>
> %{dynrel} and build changelog info are a build property. They are
> computed at build time:
> – if computed ev ≠ last saved ev or last %{dynrel} is undefined
> → reset %{dynrel} to 1
> – otherwise if computed neutralized-r = last neutralized-r
> → bump %{dynrel}
> – otherwise, do nothing, something already took care of things
> – save the new computed ev, computed neutralized-r and dynrel to the
> detached files
> – add a build line with the full evr (and full dist) to detached
> changelog
> – at build time, there is no relationship with magic git state or
> magic buildsys info, the srpms are 100% autonomous as before. The
> detached changelog has already been populated manually or by distro
> automation or manual processes when it reaches build stage and a new
> build is recorded (with a new dynrel value, if necessary).
>
> > Is the intention here that with each new build of the same package,
> > the value of %{dynrel} is bumped and committed back?
>
> It is bumped (or reseted to 1) whenever comparison with last saved
> state shows a bump or reset is needed.
>
> Build-time state changes need to be commited back, of course (otherwise
> the produced srpm needs to be re-imported manually)
>
> > That means the evr file read from the repository needs to contain
> > somehow outdated values at this point (when srpm is being built in
> > build system), otherwise %{dynrel} would be always bumped?
>
> Final dynrel state is not computed before a build, yes. The build will
> decide if it needs to bump dynrel, or can reset it to 1. I had
> forgotten to document the reset, sorry.
>
> There is no need to compute a new dynrel before actual build, the
> packager may yet change ev or r.
>
> > > – a build line is added to the detached changelog, using the
> > > current
> > > date and full evr (without %{dist} neutralization)
> > > — that probably requires defining a rpm variable to track whom
> > > the
> > > build is attributed to
> > > – it can default to "Anonymous coward" or whatever if not
> > > explicitely
> > > set
> >
> > I think today's changelogs do not contain name of the person who
> > built the pacakge
>
> Today’s changelog mixes who made spec changes and who built the result.
> That what confuses you. Confusion is anthetical with automation. This
> proposal clearly documents that changelog can change at build time
> (fedpkg build) and between builds (fedpkg commit).
>
> From a package consumer POW, the only attribution and timestamp that
> matters is who build the final package and when, because the builder is
> taking the responsability of pushing a result to users.
The person that was responsible for compiling some set of changes into
the release and pushed those changes into src.fp.o carries
responsibility too and I would say it is the person one should contact
if something doesn't work because this person knows the most about the
package. (I am assuming this person would be outsourcing the job of
actually launching the build to someone else - that someone else
doesn't need to know anything except of the commit id to build).
But okay, basically you are saying that for dynamic release, we don't
need any git context and we just need a build-counter, which is being
maintained by build system and committed back when incremented.
What about fedpkg local and fedpkg verrel then? Either, they will be
giving you always "old" %{dynrel} because when you actually invoke
fedpkg build you will get a bump on %{dynrel} or they will be giving
you always the "future" value - i.e. they will always assume you are
only about to call `fedpkg build`. That would work until you invoke
fedpkg build. If you add new commit after wards and call `fedpkg
local` or `fedpkg nvr`, you will be suddenly getting "old" %{dynrel}
value in Release. I think it would be better from user standpoint of
view if fedpkg verrel and fedpkg local give you always the correct NVR
value - i.e. the exactly the value that you will get when you send the
current commit into build system.
As for changelog, generally with build system commiting back to
dist-git, there is a problem that I can be concurrently pushing
changes while the build system tries to push its changes and build
system is not human so it will not know how to resolve such situation.
Do you have a solution for situation when I launch a build and then I
do another change, commit, and push (the changes to changelog can be
quite arbitrary here)? I guess it would require commit ids at places
where the build info line should be inserted?
I think for simplicity, there should be only one owner of the
changelog file, otherwise you need to solve synchronization issues but
maybe you have some nice simple solution for this so that build system
doesn't need to solve merge conflicts on its side...
>
> A clean changelog is:
>
> %changelog
> * timestamp builder-id - built-EVR ← set at build time
> - change ← set at commit time
> - change ← set at commit time
> - change ← set at commit time
>
> Not sure at all the change lines need attribution or timestamping. From
> a package user POW they are virtual potential changes that won’t
> materialize before a build.
>
> So yes before someone presses the build button the detached changelog
> will look like this
>
> %changelog
>
> - staged change
> - staged change
> - staged change
>
> * timestamp builder-id - built-EVR ← set at build time
> - past change
> - past change
> - past change
>
> The act of building adds the final %changelog line:
> * timestamp builder-id - built-EVR
>
> > You mean there would be different kinds of built srpms?
>
> We already have two level of srpms, one before dynamic BuildRequires
> processing and one after. It’s up to rpm upstream to decide at which
> srpm level they would prefer preforming dynrel and dynchangelog
> munging.
I think, this is not a decision of rpm upstream but an infrastructure
thing or a mock thing. It's true that mock is under in
rpm-software-management group but under "rpm upstream" I really
imagined this particular repository:
https://github.com/rpm-software-management/rpm but you probably meant
the whole rpm-software-management group.
>
> I want upstream rpm involved for dynrel and dynchangelog to be
> independent from current Fedora git infra, and interoperable.
>
> > For copr, it is not possible, because it has read-only access to git
> > repositories being built.
>
> It has some access to private git repos, you see copr initialising them
> when giving it a new srpm to build.
True, but these repositories are read-only for outside world and it's
a copr's implementation detail that they exist at this point. The
source repositories are e.g. src.fp.o, pagure.io, github.com,
bitbucket.org, etc.
I think your proposal wouldn't quite work for copr because it has no
access to those repositories (which especially for src.fp.o is good in
your proposal, otherwise copr would be modifying src.fp.o repos).
So If someone sets %{use_dynstate} in those external repos and uses
%{dynrel} in Release, copr _can_ bump it in the way you described
because the evr file and the file storing %{dynrel} (effectively build
counter) can be put into those private repos.
So if someone wanted to have a build counter in the release, yes, this
could be an implementation although it would be much easier to simply
catch the information from copr's database where this info is stored
instead of manipulating files in copr-dist-git to provide the next
value of the build conter.
But with changelog, the situation is different because copr couldn't
push the updated changelog with build info back to the source repo so
in the source repo, there would be always some kind of incomplete
changelog file ... in fact there would never be a changelog file at
all because nobody uses fedpkg commit in git repos that are not from
src.fp.o so this dynamic changelog thing wouldn't be usable for copr
at all.
>
> > I think there are good examples where some things can be done without
> > rpm changes - e.g. the simple-koji-ci to do automatic scratch builds
> > on a new commit.
>
> For something that is effectively a format change (moving changelog
> outside the spec itself) it’s a bad bad idea to do it without rpm
> upstream vetting and involvment. Sure you can kludge things without
> them. Operational word: kludge. Will forever be a pain point as long as
> it is not integrated upstream. Magic git-only or bbuildsys-only state
> is a misfeature, (s)rpms should cut the git/buildsys cordon as soon as
> they are born.
>
> scratch build are something else entirely. They are transient non-
> shared things that do not need to cleanly export or import in other
> build systems
>
> Regards,
>
> --
> Nicolas Mailhot
> _______________________________________________
> devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
