On Thu, 25 Jun 2020 at 05:29, Dan Čermák <dan.cermak@xxxxxxxxxxxxxxxxxxx> wrote: > > Hi Stephen, > > this will probably get buried given the immense amount of replies in > this thread already, but nevertheless, here's my 2cts. > > Stephen John Smoogen <smooge@xxxxxxxxx> writes: > > > On Sat, 20 Jun 2020 at 17:42, Neal Gompa <ngompa13@xxxxxxxxx> wrote: > >> > >> On Sat, Jun 20, 2020 at 5:25 PM John M. Harris Jr <johnmh@xxxxxxxxxxxxx> wrote: > >> > > >> > On Saturday, June 20, 2020 4:42:17 AM MST Neal Gompa wrote: > >> > > TL;DR benefits of modularity for Fedora: > >> > > > >> > > * Automating build chains for producing artifacts > >> > > * Straightforward mechanism of producing non-rpm artifacts using our > >> > > existing tooling (modules -> flatpaks/containers/etc.) > >> > > >> > Both of these have nothing to do with Modularity, and can be done with > >> > existing RPMs. > >> > > >> > >> They have everything to do with Modularity, because that layer is > >> where that stuff was implemented. Modularity was the result of the > >> efforts involved with Factory 2.0, which gave us a lot of improvements > >> in our build infrastructure tooling for the first time since 2007. > >> Most of that rolled out in 2017, a full ten years after the last > >> revamp of our infrastructure. > >> > > > > I think that John and others aren't aware of how a module is built > > enough to understand what you meant by > > * Automating build chains for producing artifacts > > compared to how it is done normally. > > > > In normal times, a packager goes through a list of packages, updates > > spec files with new tags and rebuilds them one by one as needed.. > > sometimes multiple times because of bootstrapping or finding out that > > the order you tried was wrong. A made up example from my days of doing > > this for a different place (this isn't what is needed anymore but long > > ago I had something similar): > > bison > > flex > > gcc [options 1] > > bison > > flex > > gcc [options 2] > > glibc > > bison > > flex > > gcc > > > > do them in one order and the apps came out working... do them in the > > wrong order and it might not. Rust, Java and other language stacks > > have similar loops. A packager may have to coordinate with multiple > > people to do this several times. > > > > In a module, you write that all down in the manifest with the order > > that you want packages built in and if you need to loop through them > > with different options. Then MBS does the builds in an automated > > fashion and it is repeatable. To me this is the biggest win here as > > for various groups of mass-rebuilds it cuts down errors when order > > matters and you have to do multiple ones to get from package set A to > > package set A+1. > > This is actually freaking awesome! The one thing where our build system > is really laking is that we have to manually figure out the build order > and rebuild stuff manually. Other systems like OBS handle this > completely automatically, albeit bootstrapping on OBS has its issues as > well. > > Anyway, is there any chance that we can get this functionality split out > of MBS and add it to koji? If we then convince koschei to kick of real > builds in Rawhide, then we'd be in a much better shape build system wise > imho. > Not without packagers en-mass accepting changes to how 'THEIR' packages are built. That middle gcc needs to have a different RPM release than the one before it and the one after it. That means that in our current work flow you could start working on a package and between your starting on updating a .spec file and committing and trying to push it.. it automatically got bumped several above because someone else was doing these builds to get 'their' package built correctly. This has been the gordian knot of our package ownership/maintenance.. with various packagers reverting and undoing things in the middle of someone else because THEIR package got updated in a way they didn't want. This has been fought over since the first days of the Fedora.us project and honestly never seems to go away. The political solution was to assume that in a module you take responsibility for all the packages in that module even if other people 'own' their 'non-modular' versions. You can then do what is needed to the packages and then not publish those special middle versions of gcc which you needed but would have broken normal gcc for everyone else.. or some other odd thing. [This again is all made up examples.. it could be a language which seems to need you to build each version only from a specific previous version and so you need to chain an 'infinite' list to bootstrap to a newer version.] I honestly do not know enough about OpenSuSE Build System to know how they deal with this automatically. I had assumed it wasn't too automatic but comes from how detailed Suse spec files are so that you don't have a package solver 'guess' what was needed. That simplifies the dependency graph a lot. Also that packagers do not 'own' things in the same way they have in Fedora land so the politics is skipped. -- Stephen J Smoogen. _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
