Can I make a strong suggestion/plea here? Could you meet with the folks working on rawhide multibuild gating and confirm all that process works with your change and then modify your change to just say 'requires multibuild rawhide gating' and only include in your change the things that are over/above/build on that? Right now there is a ton of overlap, but I see places (see below) where it's different/opposing, so the two changes would break each other. In most cases I would think it would be/could be additive thought. On Fri, Oct 18, 2019 at 02:09:14PM -0400, Ben Cotton wrote: > https://fedoraproject.org/wiki/Changes/OnDemandSideTags > > see also: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx/thread/4AS3PN23TOCBROA4RZN4TNDBZOP4VE2G/ > > = On-demand Side Tags = > > == Summary == > Allow on-demand side tags, and allow packagers to a) tag whatever rpms > as build dependencies into those side tags, b) build packages they > want in those side tags, and then c) tag all or some of those new > packages back into appropriate “main” tag (e.g. > fNN-updates-candidate). Multibuild rawhide gating does a lot of this already, but doesnt let people tag arbitrary things around. You can make a side tag, you can build things in it, you can remove a side tag (without merging it), or you can request merging it (via bodhi). All this work is already done and being tested. The about sounds very free for all to me. "whatever rpms" "all or some". From any branch? anytime? It should definitely NOT include scratch builds or deleted builds. > > This new process will allow packagers to build rpms using older or > newer versions of build dependencies than the buildroot in a given > Fedora release, providing a subset of Modularity features in an > additive way (no influence on the way dependent packages are built or > installed). It may also coexist with Modularity in its current form. _only_ as build dependencies? What enforces that? ...snip... > The second part of Modularity is simply removed: '''users only get > normal rpms'''. After rpms have been built, some or all of them are > merged into the normal tags and delivered to users as usual. This is > pretty much the same as the usual rebuild of specific language or > graphic stacks in side tags that we do already. Dependencies between > rpms must be expressed using > Requires/Recommends/Provides/Suggests/Conflicts. Do note that Conflicts are really poor for the end user. Would this adjust or change any auto requres/provides? > The first part is replaced by a simpler scheme: '''we don’t define any > specific syntax or repositories to build packages'''. The packager has > to issue a sequence of 'fedpkg tag-build', 'fedpkg build', and 'koji > wait-repo' commands as appropriate (or 'fedpkg chain-build' or …). The > packager ''may'' link an older rpm into the side tag, allowing > building against old versions of packages. This means that if the > packager is not a committer to the package, they may only use a > version which was previously built in koji. The packager selects which > rpms should be merged back into the main tag. Old packages obviously > cannot be merged back. How can a maintainer tell what packages they need to merge back? How can they tell when an 'old' package cannot be merged back? They can pick from anything ever built? What if they pick versions that are insecure or have major bugs in them? > > To minimize disruption for users and current packagers, modules are > required to be opt-in only. There are no default modules and > non-modular rpms cannot depend for build or installation on modular > rpms. This way there is less burden to other packagers and Modularity > can evolve more easily. > > === Implementation === > > '''Ability to create side tags “on demand”, i.e. with an automated API > call''' must be implemented. Koji and other infra tools would need to > be able to deal with a large number of tags. Modularity already > creates a large number of tags, so this should not be a significant > change. This is already implemented by rawhide gating folks. > > Packager tools like fedpkg must grow '''new commands to create side tags'''. This is already done and merged by rawhide gating folks. > === Version bump in a language stack === > > It is determined that a new set of rust rpm versions should be built. > A member of the rust-sig bumps the versions in dist-git for all > relevant packages, creates a side tag, and does the huge chain-build. > Once finished, this side tag is merged into rawhide-build. If desired, > it may also be merged into f31-updates-candidate and/or > f30-updates-candidate and bodhi updates submitted. But to be clear this works on older/stable releases because rust only produces static binaries right? this would not be the case for other stacks at all. > rust-sig already has tooling to generate the appropriate build-order > for modularity’s .yaml file. The same code can be reused to generate > the order for the chain build. > > === Three streams of a database package === > A security patch needs to be applied to three versions of postgresql. > There are postgresql, postgresql82, postgresql81 dist-git repos. The > patch is added in dist-git for each of the packages, and they are > rebuilt as appropriate. Since we want to build each package with > different compiler versions, the packager uses a bash loop to schedule > separate builds of each of the three packages in each of the supported > Fedora versions. This is assuming someone went to the trouble of creating those parallel installable packages already. > > (Once we have automatic creation of bodhi updates from koji builds, > the packager will not have to do anything more. Right now they still > need to create one big update that will then be split into separate > updates for each Fedora release.) > > A small advantage compared to current modules is that packages are not > built against an already obsolete Fedora base, so users get the > benefit of toolchain improvements. I thought modules also build against whatever platform they want. I guess optionally they could build against one and ship to all releases, but again that only works for very static things right? ...snip... > > === Package needs some old dep to build === > > Poor packager needs gcc-9.0, at the time where gcc-9.1 is the default > in the build root, to build koo. They bump the version of koo in > dist-git, create a side tag, tag old gcc-9.0-12.fc31 build into the > tag, build koo, merge the built package back into f32-build and > f31-updates-candidate. As long as there isn't a libgcc requires there I guess. So for this case you would need the bodhi sidetag merging to know what is and isn't a buildroot only package. > === Package needs some old dep to install === > > koo-gui is only compatible with python3-qt5-5.12 when python3-qt5-5.13 > has been released, and cannot be built with the new version. koo > maintainers may either a) arrange for python-qt5_5_12 compat package > to be created and then use it in the build root and as installation > dep, b) install old python3-qt5-5.12 package in build root, and bundle > some contents of that package into its output rpms, and add > Provides:bundled(python3-qt5)=5.12. > > === FAQ for “other developers” === > > ==== I maintain several modular streams of my package additionally to > the default version in Fedora. Can I keep doing this? ==== > > Yes, you can keep any alternate version in modular branches. When > appropriate, consider compat packages that can often be > co-installable, unlike modular streams. > > ==== I maintain my packages as modular only. Can I keep doing this? ==== > > Yes, as long as those modules are “leaves”, in the sense that no > non-modular rpms depend on them. But users may not 'see' them unless they know to module enable... > ==== I would like to use an rpm which is currently only available as > module as a build dependency ==== > > The owner of the module should be asked to demodularize the module > (and either convert it to fully traditional rpms, or build some rpms > using the new side-tag process). Those rpms can then be used as build- > and installation-time dependencies. > If the owner of the module is not interested in providing non-modular > rpms, other packages may add such rpms to the distribution (and again > build them in one of the two ways). > > ==== Which rpms can be used as build dependencies in the side tag? ==== > > In principle, any rpm which is available in koji (including the > recursive dependencies, so it can actually be installed). In addition, > the packager may build rpms and use them in the side tag build root. > In practice, we expect that packagers will use one of the Fedora > releases (with updates), as the base. This would need the sidetag work to allow arbitrary tagging. It would still need to avoid some things however (scratch builds, deleted builds, etc). ...snip... I am likely missing something here, but it sounds to me like the big downside here is that if you want to provide multiple versions of your application you would still have to re-work things to make them parallel installable (which in some cases is super easy, like postgres, but in others is really really hard). kevin
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
