On Thu, Dec 17, 2020 at 8:06 PM Ben Cotton <bcotton@xxxxxxxxxx> wrote: > > https://fedoraproject.org/wiki/Changes/Enable_Spec_File_Preprocessing > > > == Summary == > This change should enable an opt-in spec file preprocessor in Fedora > infrastructure for the benefit of packagers. The preprocessor allows > some very neat tricks that were impossible before, for example > generate changelog and release automatically from git metadata or pack > the entire dist-git repository into an rpm-source tarball (effectively > allowing unpacked repos to live in DistGit). As far as I can tell, this is the third implementation of generated changelogs ... did the autorelease / autochangelog work that was even already deployed in staging not go anywhere? > == Owner == > * Name: [[User:clime| Michal Novotný]] > * Email: clime@xxxxxxxxxxxxxxxxx > > > == Detailed Description == > > There is a recently added feature into mock: > [https://github.com/rpm-software-management/mock/wiki/Plugin-rpkg-preprocessor > the rpkg preprocessor] which, if enabled, introduces an intermediate > step just before srpm building. This step consists of running the spec > file through a text preprocessing engine that includes an already > present library of macros designed specifically for rpm spec file > generation from git metadata. This library is called > [https://docs.pagure.org/rpkg-util/v3/macro_reference.html > rpkg-macros]. The macros there allow packagers to have their > `%changelog`, `Release`, `Version`, `VCS` tag, or even `Source` fields > automatically generated from dist-git repository data and metadata. > The library can be easily extended in future to support more packager > use-cases or even a completely new library can be developed that > doesn't look at git metadata at all and instead, for example, analyses > already present tarball content to render spec file based on upstream > information. This doesn't mean it will happen but the framework is > generic enough to support that. There is also support for user-defined > macros that are loaded on-demand from a file placed alongside the > package sources, maintained by packager. This feature wouldn't be > enabled by this change from start but it's an example of freedom that > the preprocessing framework is able to provide. Enabling this change > should be very easy, basically adding: > > <pre> > config_opts['plugin_conf']['rpkg_preprocessor_enable'] = True > </pre> > > into mock configuration of Koji builders and using at least mock 2.7. > Some very minor change may be also needed in Koji regarding the spec > file lookup. > > Even if the change is enabled on the infrastructure level like this, > the packager will still need to opt-in to use the preprocessor. The > opting-in is done by placing `rpkg.conf` file into the package > top-level directory with the following content: > > <pre> > [rpkg] > preprocess_spec = True > </pre> > > When this is done by a packager, the preprocessor will be finally > enabled for the given package. > > Alongside, there is an ongoing work to add the preprocessor support > into the `rpkg` python library so that a packager can easily work with > the spec files containing the preprocessor (rpkg) macros: > https://pagure.io/rpkg/pull-request/530 > > The macros are supported since epel6 until the most latest Fedora > (preproc, rpkg-macros, and rpm-git-tag-sort packages are needed). The > spec preprocessing step in mock happens in a target chroot just before > the srpm build. > > > == Benefit to Fedora == > > This change offers solution to some long-standing issues in Fedora > around packaging (i.e. automatic release and changelog generation) > while also offering some interesting future options (for example > unpacked dist-git repos). The big advantage of this approach is that > it is explicit. Spec file stays the source of truth and by looking > inside one, you will be able to determine how the text will expand for > a certain git repository state. What does "unpacked dist-git repos" mean? Is this a euphemism for source-git? > == Scope == > * Proposal owners: > For the very basic support, probably a small patch in Koji is needed > to be able to lookup not only `.spec` files but also `.spec.rpkg` > files (the `.spec.rpkg` extension explicitly states that the spec file > is a template). Also the `rpmdevtools/rpmdev-bumpspec` script should > be tweaked to be compatible with spec files using the macros. Is the Change owner going to submit patches for fedpkg and rpmdevtools? > * Other developers: > Some optional help with `rpmdevtools/rpmdev-bumpspec` changes would be welcome. rpmdev-bumpspec is a tangled mess of spaghetti code and I'd rather not touch it or make it even more complicated. I also think this can't be optional. releng uses rpmdev-bumpspec as part of scripted mass rebuilds, so it must work with all packages. > * Release engineering: [https://pagure.io/releng/issue/9910 #9910] (a > check of an impact with Release Engineering is needed) > Enabling the rpkg_preprocessor plugin in mock config for Koji builders. > > * Policies and guidelines: > The new macro support should be mentioned or even described in the > packaging guidelines. We should decide if the full power of the > rpkg-macros library should be allowed from start (i.e. even unpacked > repos). > > * Trademark approval: N/A (not needed for this Change) > > * Alignment with Objectives: N/A > > == Upgrade/compatibility impact == > Because of the opt-in nature on packager side, there should be no > compatibility issues. > > == How To Test == > Once the feature is enabled, one can test it by providing the > `rpkg.conf` file with the required content in a package repository and > use some rpkg macro in the spec file: e.g. > > <pre> > Name: {{{ git_dir_name }}} > </pre> > > to generate the name of the package from the repository name (this > should actually produce the original text as package names should be > the same as the repository basenames). Not sure I understand, but what's the benefit of making the Name a macro determined by the repository name? As stated, they must always be the same anyway, so why make this a dynamic templated value? > To try it out first without committing to dist-git, one can use `rpkg` > command-line tool from > https://copr.fedorainfracloud.org/coprs/clime/rpkg-util/ or even > fedpkg's koji scratch build after > [https://pagure.io/rpkg/pull-request/530 the work in the pyrpkg] > library is finished. > > One can also currently use Copr's SCM "rpkg" build method where the > macros are enabled but the rpkg-macros there are in version 2 whereas > this change is about introducing the > [https://docs.pagure.org/rpkg-util/v3/macro_reference.html version 3 > rpkg-macros]. However, while there are some differences between v2 and > v3, the idea and most of the working is the same. > > == User Experience == > This change is intended for packagers. It should help to make a bit of > their work easier and offer them some new interesting options. > > == Dependencies == > N/A > > == Contingency Plan == > > Packagers can opt-out on individual basis by removing the `rpkg.conf` > file or just setting the `preproces_spec` property to `False`. On > infrastructure level, the rpkg_preprocessor plugin could be disabled > again. > > == Documentation == > - [https://github.com/rpm-software-management/mock/wiki/Plugin-rpkg-preprocessor > Mock's rpkg_preprocessor plugin] > > - [https://docs.pagure.org/rpkg-util/v3/macro_reference.html > rpkg-macros reference (the library of macros ready to be used in spec > files)] > > == Release Notes == > Currently N/A > > > -- > Ben Cotton > He / Him / His > Senior Program Manager, Fedora & CentOS Stream > Red Hat > TZ=America/Indiana/Indianapolis > _______________________________________________ > devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx > To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
