On Thu, Dec 17, 2020 at 3:15 PM Fabio Valentini <decathorpe@xxxxxxxxx> wrote: > > On Thu, Dec 17, 2020 at 8:06 PM Ben Cotton <bcotton@xxxxxxxxxx> wrote: > > > > https://fedoraproject.org/wiki/Changes/Enable_Spec_File_Preprocessing > > > > > > == Summary == > > This change should enable an opt-in spec file preprocessor in Fedora > > infrastructure for the benefit of packagers. The preprocessor allows > > some very neat tricks that were impossible before, for example > > generate changelog and release automatically from git metadata or pack > > the entire dist-git repository into an rpm-source tarball (effectively > > allowing unpacked repos to live in DistGit). > > As far as I can tell, this is the third implementation of generated > changelogs ... did the autorelease / autochangelog work that was even > already deployed in staging not go anywhere? > That effort, as far as I'm aware, was waiting for the staging Koji to be restored to finish the work, which it was recently. I believe those developers are interested in getting this in place in production Koji. > > == Owner == > > * Name: [[User:clime| Michal Novotný]] > > * Email: clime@xxxxxxxxxxxxxxxxx > > > > > > == Detailed Description == > > > > There is a recently added feature into mock: > > [https://github.com/rpm-software-management/mock/wiki/Plugin-rpkg-preprocessor > > the rpkg preprocessor] which, if enabled, introduces an intermediate > > step just before srpm building. This step consists of running the spec > > file through a text preprocessing engine that includes an already > > present library of macros designed specifically for rpm spec file > > generation from git metadata. This library is called > > [https://docs.pagure.org/rpkg-util/v3/macro_reference.html > > rpkg-macros]. The macros there allow packagers to have their > > `%changelog`, `Release`, `Version`, `VCS` tag, or even `Source` fields > > automatically generated from dist-git repository data and metadata. > > The library can be easily extended in future to support more packager > > use-cases or even a completely new library can be developed that > > doesn't look at git metadata at all and instead, for example, analyses > > already present tarball content to render spec file based on upstream > > information. This doesn't mean it will happen but the framework is > > generic enough to support that. There is also support for user-defined > > macros that are loaded on-demand from a file placed alongside the > > package sources, maintained by packager. This feature wouldn't be > > enabled by this change from start but it's an example of freedom that > > the preprocessing framework is able to provide. Enabling this change > > should be very easy, basically adding: > > > > <pre> > > config_opts['plugin_conf']['rpkg_preprocessor_enable'] = True > > </pre> > > > > into mock configuration of Koji builders and using at least mock 2.7. > > Some very minor change may be also needed in Koji regarding the spec > > file lookup. > > > > Even if the change is enabled on the infrastructure level like this, > > the packager will still need to opt-in to use the preprocessor. The > > opting-in is done by placing `rpkg.conf` file into the package > > top-level directory with the following content: > > > > <pre> > > [rpkg] > > preprocess_spec = True > > </pre> > > > > When this is done by a packager, the preprocessor will be finally > > enabled for the given package. > > > > Alongside, there is an ongoing work to add the preprocessor support > > into the `rpkg` python library so that a packager can easily work with > > the spec files containing the preprocessor (rpkg) macros: > > https://pagure.io/rpkg/pull-request/530 > > > > The macros are supported since epel6 until the most latest Fedora > > (preproc, rpkg-macros, and rpm-git-tag-sort packages are needed). The > > spec preprocessing step in mock happens in a target chroot just before > > the srpm build. > > > > > > == Benefit to Fedora == > > > > This change offers solution to some long-standing issues in Fedora > > around packaging (i.e. automatic release and changelog generation) > > while also offering some interesting future options (for example > > unpacked dist-git repos). The big advantage of this approach is that > > it is explicit. Spec file stays the source of truth and by looking > > inside one, you will be able to determine how the text will expand for > > a certain git repository state. > > What does "unpacked dist-git repos" mean? Is this a euphemism for source-git? > > > == Scope == > > * Proposal owners: > > For the very basic support, probably a small patch in Koji is needed > > to be able to lookup not only `.spec` files but also `.spec.rpkg` > > files (the `.spec.rpkg` extension explicitly states that the spec file > > is a template). Also the `rpmdevtools/rpmdev-bumpspec` script should > > be tweaked to be compatible with spec files using the macros. > > Is the Change owner going to submit patches for fedpkg and rpmdevtools? > > > * Other developers: > > Some optional help with `rpmdevtools/rpmdev-bumpspec` changes would be welcome. > > rpmdev-bumpspec is a tangled mess of spaghetti code and I'd rather not > touch it or make it even more complicated. > I also think this can't be optional. releng uses rpmdev-bumpspec as > part of scripted mass rebuilds, so it must work with all packages. > I agree that this is absolutely not optional. > > * Release engineering: [https://pagure.io/releng/issue/9910 #9910] (a > > check of an impact with Release Engineering is needed) > > Enabling the rpkg_preprocessor plugin in mock config for Koji builders. > > > > * Policies and guidelines: > > The new macro support should be mentioned or even described in the > > packaging guidelines. We should decide if the full power of the > > rpkg-macros library should be allowed from start (i.e. even unpacked > > repos). > > > > * Trademark approval: N/A (not needed for this Change) > > > > * Alignment with Objectives: N/A > > > > == Upgrade/compatibility impact == > > Because of the opt-in nature on packager side, there should be no > > compatibility issues. > > > > == How To Test == > > Once the feature is enabled, one can test it by providing the > > `rpkg.conf` file with the required content in a package repository and > > use some rpkg macro in the spec file: e.g. > > > > <pre> > > Name: {{{ git_dir_name }}} > > </pre> > > > > to generate the name of the package from the repository name (this > > should actually produce the original text as package names should be > > the same as the repository basenames). > > Not sure I understand, but what's the benefit of making the Name a > macro determined by the repository name? > As stated, they must always be the same anyway, so why make this a > dynamic templated value? > > > To try it out first without committing to dist-git, one can use `rpkg` > > command-line tool from > > https://copr.fedorainfracloud.org/coprs/clime/rpkg-util/ or even > > fedpkg's koji scratch build after > > [https://pagure.io/rpkg/pull-request/530 the work in the pyrpkg] > > library is finished. > > > > One can also currently use Copr's SCM "rpkg" build method where the > > macros are enabled but the rpkg-macros there are in version 2 whereas > > this change is about introducing the > > [https://docs.pagure.org/rpkg-util/v3/macro_reference.html version 3 > > rpkg-macros]. However, while there are some differences between v2 and > > v3, the idea and most of the working is the same. > > > > == User Experience == > > This change is intended for packagers. It should help to make a bit of > > their work easier and offer them some new interesting options. > > > > == Dependencies == > > N/A > > > > == Contingency Plan == > > > > Packagers can opt-out on individual basis by removing the `rpkg.conf` > > file or just setting the `preproces_spec` property to `False`. On > > infrastructure level, the rpkg_preprocessor plugin could be disabled > > again. > > > > == Documentation == > > - [https://github.com/rpm-software-management/mock/wiki/Plugin-rpkg-preprocessor > > Mock's rpkg_preprocessor plugin] > > > > - [https://docs.pagure.org/rpkg-util/v3/macro_reference.html > > rpkg-macros reference (the library of macros ready to be used in spec > > files)] > > > > == Release Notes == > > Currently N/A > > I'm generally not excited about this, as it adds a huge layer of indirection and a ton of extra magic that makes it harder to decipher what is happening. -- 真実はいつも一つ！/ Always, there's only one truth! _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
