Hi, Thank you for this very useful summary. One general problem with the thinking behind this is that it applies much more to CentOS or RHEL than it does to Fedora. In particular: > users want a solid, stable, reliable, *unchanging* system. In that case, really, Fedora is not the answer. No matter how much I love Fedora, I know that it is never going to be most stable and reliable, and it is never going to be *unchanging*. The same is true for the idea that multiple parallel versions of packages will be supported. Various statistics show that we simply don't have the human resources and will to do this: there are almost no karma updates in "oldstable" (meaning e.g. F29 at the time when F30 has been released), the number of bugs fixed for "oldstable" is generally low, modules with more than one version are a minority, etc. Those two things are different in RHEL, first because people are paid to do certain jobs, but primarily because that is the focus of such long-term distributions. In Fedora, I don't think we have "customers" who would even use this that much, even if we provided it. If we assume that long-term maintenance is not our focus, and we don't have the resources to do it anyway, what is Modularity used for in Fedora? In current practice, it is used mostly as a glorified buildroot-management mechanism. But I think we have or can build better alternatives for this. Multi-build gating in rawhide is close to being released. Based on this functionality, we can build solutions which build on the current distribution in a more incremental way. The biggest advantage is that those solutions are only at the build side, so they don't "leak" into the UX. > Sometimes, a package is needed only to build the software and is not > required at run-time. In such cases, Modularity should offer the ability to > keep those build-time dependencies entirely private and not exposed to the > Fedora Package Collection at large. ... > *Requirement*: Packagers must be able to encode whether their output > artifacts are intended for use by other projects or if they are effectively > private to the alternative version. Packagers must also have a way of > finding this information out so they understand what they can and cannot > rely on as a dependency. > *Requirement*: Build-time only dependencies for an alternative version may > be excluded from the installable output artifacts. These excluded artifacts > may be preserved by the build-system for other purposes. This is another thing that makes sense for commercial distro, but not for Fedora. Kevin Koffler was arguing this very passionately, and I think his critique is spot on: this should be an anti-goal for Fedora. If Modularity ever become popular, we'd have a collection of fiefdoms of incompatible packages used to built particular leaf packages. Having a rich ecosystem of dependencies is one of the best things about Fedora. I know it is often inconvenient to have to package something, but in the long run it pays off. Every time I put in an effort to update some dep I couldn't care less about, I know somebody will do the same with another dep that I'll need next week. Instead of privitizing dependencies, we should turn more to SIGs to manage subcollections. In particular, the Python SIG, the Stewardsip SIG, the Neuroscience SIG, etc, etc, are doing great job with managing huge trees of packages. We should also use more automatic building of some packages, like packit and automatic updates of language stacks. (And what would it even mean to have a private package: the truth is that the only thing a maintainer needs is to go through initial review and then keep the package non-FTBFS. But if they use it in a Module, they have to do this too. And since is everything is public and there is no guarantee of support, making things "private" is only about making packages harder to obtain and install. Another non-goal.) > First and foremost, our primary driving goal is to make it easy for our > users to understand and interact with alternative software versions. In any > instance where choosing between the packager experience and the user > experience is in conflict, we elect to improve things for the user. I think Modularity (in design and in current form), falls short on this goal. This has been discussed before, but I'll resummarize: modules are not handled by the SAT solver, but are governed by a different set of rules, and will require a separate language and engine to handle module transitions. This means that every piece of software that wants to install rpms must redo the logic to handle modules. Users must learn about this logic and its quirks. If we are strictly taking about *user* POV, then having a single repo (possibly with multiple versions of some packages) is nicer than having multiple repos and/or streams that need to be managed. I'd argue that the approach that Modularity takes the easy road for packagers who want to manage a build root to build a bunch of related packages at once, but suboptimal for everyone else. > Developers often want to build their applications using the > latest-and-greatest version of their dependencies. However, that may not > have been released until after the most recent Fedora release. In > non-Modular Fedora, that means waiting up to six months to be able to work > on it there. > > *Requirement*: It must be possible to gain access to newer software than > was available at the Fedora release GA. This is a nice goal. But we don't need Modularity to do it. Currently, the Update Guidelines discourage major-version updates in stable, for good reasons. If makes most sense to be "rolling" for packages which are backwards-compatible anyway. If we want more "rolling" packages, we should change our mindset and/or guidelines. This should be a discussion independent of the delivery mechanism. > Users are slow to adapt to changes in the way they need to behave. > Requiring them to learn a new set of commands to interact with their system > will likely result in frustration and possibly exodus to other > distributions. > > *Requirement*: It must remain possible to continue to operate with only the > package management commands used in traditional Fedora. We may provide > additional commands to support new functionality, but we must not break the > previous workflow. If that is a goal, let's make the buildroot-level changes invisible to users, i.e. even if stuff is built in a non-standard way, deliver only normal rpms. > Since the earliest days of Linux, the “package” has been the fundamental > unit of installable software. If you want to have some functionality on the > system, you need to learn the name of the individual packages that provide > that functionality (not all of which are named obviously). As we build > modules, one of the goals is to try to focus installation around use-cases > rather than around upstream projects. A big piece of this is that we want > to have a way to install a subset of a module that supports specific > use-cases. A common example being “server” and “client” cases. > > *Requirement*: It must be possible to install a subset of artifacts from an > alternative version. These installation groups should be easily > discoverable. You mean like openssh-server and openssh-clients, postgresql and postgresql-server, every libfoo and libfoo-devel? (Or any meta-package used to pull in a bunch of things…) > *Requirement*: It must be possible for alternative versions of software to > go end-of-life during a Fedora release. This does not mean that the > software must disappear from the repositories, merely that an assertion > exists somewhere that after a certain date, the package will not receive > updates. This was rejected by FESCo, and for good reasons. There is no guarantee that any package will be supported, even if people have the best intentions to do so. Thus, if a packager knows that they won't be able to support a package in some release, they should orphan the package in that branch. Either somebody will pick it up, or it will be retired. And if they stop maintaining a package in a released version, hopefully somebody will pick it up. Declaring EOLs for packages is useless in Fedora. > Some third-party add-on repositories (particularly EPEL) have been limited > in the past by relying on the system copies of packages in the base > distribution of the release. In the particular case of EPEL, little can be > done to upgrade these system copies. In order to be able to package much of > the available FOSS software out there, it may be necessary to override some > of the content shipped in the base system with packages known to work > properly. If RHEL/Centos are creating a problem with their policies, then they should solve it too. Don't take Fedora through contortions to solve a *policy* issue in another distribution. > *Requirement*: It must be possible for third party repositories to create > alternative versions that override base distribution content at the user’s > explicit choice. > > *Requirement*: It must be possible for third party repositories to create > alternative versions of software that exist in the base distribution. copr? > Reduce duplication in packaging work > > There is plenty of software out in the wild that maintains compatibility > over time and is therefore useful to carry in multiple releases of Fedora. > With traditional packaging, this means carrying and building separate > branches of the packages for each release of Fedora. In the case of > software “stacks” which are tightly bound, this means also manually > building each of its dependencies in each release of Fedora. > > *Requirement*: It must be possible to build multiple component software > packages in the same build process. side tags? > *Requirement*: It must be possible for the packager to specify the order in > which packages must be built (and to indicate which ones can be built in > parallel). Great. We need this for buildroots and lang-stack rebuilds. Let's build automation which can figure out the build order from package dependencies. Everybody will benefit from this. Please don't ever hardcode the build order a yaml file. > *Requirement*: It must be possible to be build for all supported platforms > using the same specification and with a single build command. OK, but let's hide this in the command used to fire of builds. No need to make this visible to consumers of those packages. > Non-GoalsParallel installability > > As mentioned in the Background section, the goals of Modularity are > specifically to *not* implement parallel-installability. We recommend > instead that users should rely on other mechanisms such as virtualization > or containerization to accomplish this. If parallel-installation is > unavoidable, then Modularity is not the correct tool for this job. There are other solutions that provide parallel-installability: in particular compat packages. So the fact that Modularity can not ever provide that is a strong demerit. Please note that this also affects many more packages than obvious at first sight. Since any rpm which is provided as part of a module overrides any non-modular rpm, then some module pulling in some specific version of some small library may make huge swaths of other rpms in the distribution uninstallable. This is one of the many ways in which Modularity falls short of the declared goals of "not breaking existing workflows", "not requiring new commands", etc. > Arbitrary stream switching > > Module streams are intended to be compatible update streams. That means > they must follow the same rules regarding RPM package-level updates within > the stream. By definition, two streams of the same module exist because > upgrades (or downgrades or cross-grades…) are not capable of being done in > a safe, automated fashion. > > That does not mean that stream switching should be impossible, but it does > mean that we will not build any tools intended to handle such switching in > a generic manner. Stream switches should be handled on a module-by-module > basis and detailed instructions and/or tools written for each such case. That sounds horrible. Bad UX and a significant waste of packager time. Counter-proposal: Requires/Conflicts on individual rpms must be used to express what needs to be updated as part of a transaction. tl;dr: - don't let any buildroot-enhancement mechanisms "leak" to the delivered rpms - say "no" to private packages in Fedora - let packages follow the Fedora lifecycle as appropriate for each package, no independent EOLs - don't hardcode things that can be computed by the build system - don't build a delivery mechanism where a policy change is sufficient - express relations between packages using existing rpm deps - use coprs and side-tags Zbyszek _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
