Stephen Gallagher wrote:
> One of the recurring themes in the ongoing Modularity threads has been
> that we've made references to the problems we're trying to solve, but we
> haven't done a good job of gathering those requirements and use-cases into
> a single place. To resolve this, I've written a (fairly long) blog post
> describing the set of problems that we are trying to solve.
Unfortunately, the list of requirements is very long (IMHO, unreasonably
long), and some of the requirements are in conflict, as I am going to
explain point by point below. For some of the requirements, it is also
arguable whether they are really relevant, especially for Fedora. I am also
going to address those individually below.
I would also like to point out that none of the requirements you listed
actually require default streams, only modules for alternate versions (and
perhaps not even those), and that in fact the first requirement under
"Critical use cases for consumers" (that simplicity for users should trump
simplicity for packagers) is actually an argument against default streams.
> Please note as well that these are goals. There are numerous places where
> the implementation of Modularity at the time of this writing is not yet
> fully adherent to them.
Unfortunately, there are some requirements in your list that are mutually
exclusive (see below), so the implementation will never be fully adherent to
them, and in fact no implementation will. So some requirements will have to
go.
> This is excellent for the maintainers of the distribution, because it
> allows them to test that everything works together as a cohesive whole. It
> means that there’s one authoritative version to align to.
>
> Users, on the other hand, are most concerned about solving their problem.
> It matters less to them that the distribution is cohesive and more that
> the tools they need are available to them.
It actually matters to users that the distribution is cohesive because they
want to use more than just one application, and those applications need to
work together. And this is the area where the design restrictions of
Modularity come into play.
> The “Too Fast/Too Slow” problem is basically this: users want a solid,
> stable, reliable, *unchanging* system. They want it to stay that way for
> the life of their application. However, they also want their application
> to run using the set of dependencies it was designed for.
Do they really?
I want my applications to run using the dependencies that are part of the
distribution platform (and which, as a result, do not conflict with other
applications I may also want to install). If those dependencies are too new,
the application needs to be ported to the new versions. If they are too old,
then they probably need to be updated systemwide (assuming the newer version
is backwards-compatible, of course), as has been done with Qt more than
once.
Where that is not possible, the next best solution is to have a parallel-
installable compatibility library that provides the version required by the
application. The existing packaging guidelines for compatibility libraries
cover those pretty well: suffixed package name, the runtime packages must
not conflict (ever), the -devel packages should not conflict if it can be
avoided.
I do not care whether the library version is the exact version upstream
happened to have installed. It just needs to make the application work. And
I consider this making things work together with shared dependencies to be
the one value added by the distribution over the upstream software, i.e.,
the distribution's entire purpose.
> If that doesn’t happen to be the same version (newer or older) as the one
> selected for the monolithic distribution, the user will now have to resort
> to alternative means to get up and running. This may be as simple as
> bundling a dependency or as drastic as selecting an entirely different
> distribution that better fits their specific need.
However, Modularity does not provide a reasonable way to bundle dependencies
(that is not already possible in ursine RPMs), because as Zbigniew pointed
out, the bundled dependencies are never really private, but still pollute
both the package name namespace and the file system, conflicting with any
other version (ursine or from another module) of the same package.
The only way to prevent these conflicts is to resort to RPM-level conflict-
free bundling techniques such as static linking or the aforementioned
compatibility package pattern. But then you do not need Modularity to begin
with.
> A few years back, the Product Management team inside Red Hat performed a
> large-scale survey of customers and potential customers about the user
> experience of Red Hat Enterprise Linux. In particular, they asked about
> their level of satisfaction with the software available from the
> enterprise distribution and their opinion on these Software Collections.
>
> Perhaps unsurprisingly, the overwhelming majority of respondents were
> thrilled to have supported versions of software beyond what had shipped
> with the base operating system. What the survey team did come away with
> that was an epiphany was that the respondents generally did not care about
> the parallel installability of the SCLs. For the most part, they
> maintained individual userspaces (using bare metal, traditional
> virtualization or containers) for each of the applications they cared
> about.
And this is a very big assumption in the design of Modularity that just does
not apply in the Fedora world (and maybe not even in the CentOS world). I am
pretty sure that Fedora users typically do not maintain per-app userspaces.
It is a lot of maintenance effort to keep all those userspaces up to date
and secure. Instead, Fedora users want to install an arbitrary choice of
applications and not have them conflict. I know I do. (And, while off topic
here, the CentOS VPS I maintain is also set up like that, so this is not
limited to Fedora nor to desktop setups. All the server applications are on
one single virtual server. The host machine is a black box managed by the
hosting provider. So I administrate only one VPS userspace.)
Please keep in mind that the typical Fedora user is not an enterprise, but
an individual or a small company, where just throwing in dozens of
containers or VMs, let alone physical servers, is not going to scale.
> The most common problem reported for Software Collections was that using
> them required changes to the applications they wanted to run. SCLs install
> to a separate filesystem location from more traditional RPMs and
> applications that rely on them need to know where to look for them. (In
> SCL parlance, this is called “activating” the collection.)
>
> The consequence of this relocation on disk is that users were unable to
> take existing applications (either FOSS or proprietary) and simply use
> them. Instead, they had to modify the projects to first activate the
> collections. This was a consistent pain point.
It is clear that SCLs are also a suboptimal solution, which is why they were
never really accepted into Fedora.
The issue mainly boils down to SCLs violating (at least the spirit of) the
FHS by design. The other pain points (such as the requirement to "activate"
the collection that you pointed out) mostly follow from that.
> Given this feedback, Red Hat came to the conclusion that parallel
> installability, while nice to have, was not a critical user requirement.
But this is a non sequitur. The actual 2 issues are that 1. the
implementation of parallel installability caused problems and 2. the user
group that was queried happened to be the group not having a use for
parallel installability (which, as I explained above, is not generally
true).
I believe that any approach to parallel installability that is based on
relocating packages with minimal specfile changes (i.e., the SCL approach)
is also doomed to fail. Parallel installability should be done only where it
makes sense (i.e., it is pointless for, e.g., most leaf packages) and on a
case by case basis, taking into account the file system layout of the
individual package and honoring the FHS as well as possible without
introducing conflicts.
The compatibility libraries are good examples of that. With some trickery,
even -devel can be made to coexist (e.g., if the .so symlinks conflict, then
put them into a subdirectory of %{_libdir} and use a -L flag to link against
them). Other examples are the compatibility language interpreters or
compilers such as python* or gcc*. E.g., I have found the old RHEL/CentOS
approach of providing suffixed packages such as gcc4 with similarly suffixed
binaries (allowing to just use something like CC=gcc4 ./configure or
cmake -DCMAKE_C_COMPILER=gcc4 .) more convenient than the devtoolset SCL
approach.
> Instead, the focus would be on the parallel *availability*. By dropping
> this requirement, it became possible to create a solution that allowed the
> different versions to be swapped in and take over the standard locations
> on the disk.
Unfortunately, this approach conflicts with your later requirement that
modules can require an arbitrary version of their dependencies. The
combination of the 2 requirements introduces version conflicts that can only
be solved through containerization, which is impractical for the reasons I
already pointed out above (maintenance workload, security) and elsewhere
(system integration issues etc.), and which also has to be done manually
with no help from the tooling (and I strongly doubt that the tooling will
ever be able to automatically containerize conflicting modules in the way
that was advertised in the early Modularity presentations at DevConf in past
years, i.e., that DNF would automatically install a module as a container
rather than directly if it detects that installing it directly would
conflict, a proposed feature which, I think, is really way beyond the scope
of DNF and completely unimplementable in practice).
> Meanwhile in Fedora
>
> Of course, it’s not just Red Hat — people in Fedora are also concerned
> with solving this Too Fast / Too Slow problem for our users. Efforts
> around this kicked off in seriousness with the Fedora.next initiative
> <https://fedoramagazine.org/fedora-present-and-future-a-fedora-next-2014-update-part-ii-whats-happening/>
> and
> Fedora Project Leader Matthew Miller’s “Rings
> <https://lwn.net/Articles/563395/>” talk at the first Flock conference in
> 2013.
And already back then, there were doubts about the implementability of those
proposals.
> Critical use cases for consumers
>
> First and foremost, our primary driving goal is to make it easy for our
> users to understand and interact with alternative software versions. In
> any instance where choosing between the packager experience and the user
> experience is in conflict, we elect to improve things for the user.
As I wrote in the introduction, this is actually an argument against default
streams, which are mostly about packager experience, while making the user
experience less smooth (due to the leaky abstraction).
> Standard Locations
>
> In order to make deployment of users’ applications simpler, we need to
> make sure that software can be installed into the common, expected
> locations on the system. This includes (but is not limited to):
>
> - Libraries must be installed to /usr/lib[64].
> - Headers must be installed to /usr/include.
> - Executables must be installed to a location in the default system
> $PATH
> - Other -devel functionality such as pkgconfig files must be
> installed in their standard lookup locations.
> - Installed services may own a well-known DBUS address.
> - Services may own the appropriate standard TCP/UDP ports or local
> socket paths.
>
> *Requirement*: Installation must occur in the same locations as
> traditional RPM software delivery.
I think this requirement is too rigid. While it usually makes perfect sense
to not muck around with leaf applications to make them parallel-installable
and honor the default upstream locations, I think it is perfectly fine (and
FHS-compliant) to (e.g.) install compatibility libraries to subdirectories
of /usr/lib(64) and/or /usr/include even if the default version RPM (and the
upstream build system) installs directly into /usr/lib(64) and/or
/usr/include, in order to achieve parallel installability. IMHO, for
libraries, being able to use all versions in parallel (including the -devel
packages!) is well worth the slight hassle of having to muck with -I and -L
flags to compile software against the non-default version.
For leaf applications, modules (which just replace the default version) are
a good fit. But so are Coprs, which are pretty good at replacing leaf
packages with different versions (even older ones, with Epoch), too. For
libraries, parallel-installable suffixed compatibility packages are the
better fit.
> Don’t break the app!
>
> It is very common for Fedora to update to the latest major version of
> packages at each new semiannual release. This ensures that Fedora remains
> at the leading edge of software development, but it can wreak havoc on
> anyone trying to maintain a deployment on Fedora. If they are running an
> app that is built for PostgreSQL 9.6 and Fedora switches to carrying
> PostgreSQL 10 in the next major release, upgrading to that release may
> break their app (possibly in ways undetectable by the upgrade process).
>
> However, staying on an old version of Fedora forever has its own problems.
> Not least of these is the problem of security updates: Once a release has
> been out for about 13 months, it stops receiving errata. Moreover, new
> releases of the Fedora platform may have other useful enhancements (better
> security defaults, increased performance thanks to compiler improvements,
> etc.).
>
> *Requirement*: We need to allow users to “lock” themselves onto certain
> dependencies as long as the packager is maintaining them. These
> dependencies must continue to receive updates.
This sounds great in theory. In practice, packages are not isolated islands,
so you cannot in general just use the old application on the new system
libraries, and if the package is not a leaf, other packages in the
distribution may require the new version and not work with the old one
anymore. Sure, a lot of this can be solved given enough package maintenance
work, backporting (code and/or packaging) changes to allow the old version
to work on newer system libraries, and rebuilding dependent packages into
the module, but as others have already pointed out, manpower is already thin
enough as it stands, without this extra effort to keep old versions alive.
The "rebuilding dependent packages" approach also breaks down completely if
those same packages also need to be rebuilt by another module, because the
rebuilds will then conflict. It can be solved by putting the affected
packages into yet another, separate, module, but then that other module has
to be built against an exponential number of dependency version
combinations, as was already discussed.
> *Requirement*: There must be appropriate and helpful UX for dealing with
> when those dependencies go EOL.
The modules going EOL will always completely screw over their users, no
matter the UX. But of course, old module streams cannot be maintained
forever with the manpower that we have. So those "old version" modules are
only delaying the inevitable.
> Support the developers
>
> Developers often want to build their applications using the
> latest-and-greatest version of their dependencies. However, that may not
> have been released until after the most recent Fedora release. In
> non-Modular Fedora, that means waiting up to six months to be able to work
> on it there.
>
> *Requirement*: It must be possible to gain access to newer software than
> was available at the Fedora release GA.
This is already possible in at least 3 ways:
1. If the newer dependency is backwards compatible, we can simply update the
system version. (See, e.g., Qt, KDE Frameworks, etc.)
2. Compatibility packages can also be used for forward compatibility. (E.g.,
we already had a qt4 package back when Qt 3 was still the default qt.)
3. The newer dependency, and applications depending on it, can be provided
through a Copr.
And failing all three, up to 6 months is not really all that long to wait to
begin with.
> Additionally, Dev/Ops people are rapidly switching to a new paradigm of
> development and deployment (containers) to solve the above issue. However,
> most containers today are retrieved from public repositories. The public
> repositories are generally user-managed and have not been verified and
> validated for security.
>
> *Requirement*: Provide a mechanism for building *trusted* container base
> and application images with content alternatives.
A trusted container should contain a tested combination of packages (as
provided by a non-modular distribution release), not an untested arbitrary
mix&match of potentially incompatible (at runtime) versions.
> Keep it updated
>
> It’s not enough that other versions of software are available to install.
> They also need to be kept up to date with bug fixes and security updates.
> In non-Modular Fedora, users had the ability to force DNF to lock to a
> specific RPM NEVRA, but they wouldn’t get updates from it.
>
> *Requirement*: Alternative software must receive be able to recieve and
> apply updates.
Of course. Version-locking old package builds is never a reasonable long-
term solution. Old packages need to come from somewhere that can provide
bugfix and security updates, such as a module (relying on DNF automatically
preferring the modular version to the ursine one) or a Copr (with bumped
Epoch).
> Make it discoverable
>
> Having alternative versions available is important but not sufficient. It
> is also necessary for users to be able to locate these alternatives. Some
> of our early explorations into this area failed this ease-of-use test
> because they require the user to have knowledge of external sites and then
> to search those sites for what they think they want.
>
> *Requirement*: Users must be able to discover what alternative software
> versions are available with tools that are shipped with the OS by default.
> Ideally, these should be the same tools that they are already comfortable
> with.
This is a requirement that I also consider more rigid than really necessary.
Is it really unreasonable to require locating a Copr to get a non-default
version of something? The default version is the default for a reason, so
IMHO it should be fine if that is the only version shown out of the box, by
default.
> Don’t break existing package management workflows
>
> Users are slow to adapt to changes in the way they need to behave.
> Requiring them to learn a new set of commands to interact with their
> system will likely result in frustration and possibly exodus to other
> distributions.
>
> *Requirement*: It must remain possible to continue to operate with only
> the package management commands used in traditional Fedora. We may provide
> additional commands to support new functionality, but we must not break
> the previous workflow.
This is a noble goal, but when the bolted-on functionality modifies the
behavior of the existing tools enough to be confusing (or outright broken),
the whole masquerade breaks down. E.g., that is what happened with the
libgit2 upgrade path issue, where the magic enablement of the default stream
put DNF in a state that was not automatically upgradeable to Fedora 30,
something that would never have happened without the module and default
stream handling that was added to DNF under the hood.
> *Requirement*: Existing automation tools such as anaconda’s kickstart and
> Ansible must continue to work.
In principle, the same considerations also apply in this case, though in
this case, I am, at this time, not aware of any concrete issues caused by
the module support in those tools.
> Critical use-cases for packagers
>
> Dependencies
>
> Because very little software today is wholly self-contained, it must be
> possible for Modules to depend on each other.
>
> *Requirement*: There must be a mechanism for packagers to explicitly list
> dependencies on other software, including alternative versions. This
> mechanism must support both build-time and run-time dependencies.
Unfortunately, this requirement is inherently incompatible with the design
decision to not support parallel installation, as pointed out above. Users
(including me) will want to install different unrelated applications on the
same system (in the same userspace, not in a container, see above) without
running into random version conflicts (because those applications that look
completely unrelated to the user actually happen to transitively depend on
different versions of one particular module).
At package level, this kind of issues was known as "RPM Hell". It was an
incredibly frustrating user experience. It was (mostly) solved by
centralizing repositories and discouraging the replacement of system
packages by third-party repositories. (Better tooling, which automatically
resolves dependencies, taking them from the correct repositories rather than
some random place the user found with a search engine, and reports why a
conflict is not resolvable, also helped to some extent. But what helped the
most with minimizing those issues was the sane repository structure.) But
now, Modularity introduces something like repositories within a repository
and brings these conflicts back.
In my view, this is the biggest design flaw in Modularity.
> Alternative dependencies
>
> Some software is very restrictive about which dependencies it can work
> with. Other software may work with several different major releases of a
> dependency. For example, a user may ship two Ruby-based web applications,
> one which is capable of running on Ruby 2.5 and the other that can run on
> either Ruby 2.5 or Ruby 2.6. In non-modular Fedora, only one version of
> Ruby would be available. If the system version was 2.5, then both
> applications could run fine. But if in the next release of Fedora the Ruby
> 2.6 release becomes the system copy, one of those applications will have
> to be dropped (or patched) to work with it.
Well, in my view (and as I have already pointed out above), making things
work together, patching them where necessary (and using compatibility
packages where that is not possible), is really the whole point of a
distribution.
> *Requirement*: It must be possible to build software that can be run
> against multiple versions of its dependencies.
>
> *Requirement*: The packaging process for creating software that supports
> multiple versions of their dependencies must not be significantly more
> difficult than packaging for a single dependency.
>
> As more and more things become modules, there is concern that such things
> will grow into an unbounded matrix. For this, we need to establish
> policies on when the use of alternative dependencies is preferable or when
> it is better to constrain it to a single version or small set.
See, you point out yourself why the two requirements above are problematic.
You end up with exponentially many combinations of dependencies. Supporting
them all definitely does not scale. But supporting only a subset will
inevitably break things for users who, due to hard requirements in other
packages, need a different subset. (And that is assuming that the hard
requirements do not already by themselves conflict.)
Hence, I think creating modules (which are by design not parallel-
installable with different versions of themselves) for non-leaf packages is
a very bad idea and will inevitably lead to breakage and conflicts.
The only 2 ways to prevent those conflicts are those that I have already
pointed out: make the applications work with a common default version of the
dependency (by patching it if necessary) or ship a parallel-installable
compatibility package of the dependency for the non-default version.
> *Requirement*: Packaging guidelines need to provide advice on when to use
> multiple alternative dependencies or to select a single one.
As a result of the above, the advice provided under this requirement will
always be wrong one way or the other, you only get to choose between an
unmaintainable combinatorial explosion or broken user systems.
> Managing private dependencies
>
> When a person decides that they want Fedora to carry a particular package
> and decides to do the work to accomplish this, it is not uncommon to
> discover that the package they care about has additional dependencies that
> are not yet packaged in Fedora. Traditionally, this has meant that the
> packager has needed to package up those dependencies and then continue to
> maintain them for anyone who may be using them for other purposes. This
> can sometimes be a significant investment in time and energy, all to
> support a package they don’t necessarily care about except for how it
> supports the primary package.
But this is part of the reality of a community distribution. If your package
depends on libfoo, then you obviously have to package libfoo. So why should
others then not be allowed to benefit from libfoo for other purposes, just
because those purposes do not happen to be the one purpose you cared about?
You can always ask other packagers to comaintain libfoo if it makes sense
(e.g., because their package uses libfoo in a different way than yours).
(E.g., I comaintain kpmcore with the kde-partitionmanager (KPM) maintainer
because I maintain Calamares, which also depends on kpmcore.) But why should
they be discouraged or banned from using your libfoo (and, worse, encouraged
to bundle their own private libfoo build in their module, which then causes
conflicts when users want to use both modules)? (E.g., what should I have
done as the packager of Calamares if kde-partitionmanager (KPM) had been a
module and the kpmcore library private to that module?) It just does not
make sense.
IMHO, private dependencies should be a definite non-goal in Fedora.
> Build-time Dependencies
>
> Sometimes, a package is needed only to build the software and is not
> required at run-time. In such cases, Modularity should offer the ability
> to keep those build-time dependencies entirely private and not exposed to
> the Fedora Package Collection at large.
But WHY? The package may well not be required at runtime by your package,
but that does not mean end users will not need it, if only to compile other
packages.
Compiling software is definitely a valid use case of a Free Software
distribution. We ship compilers, -devel subpackages, etc. for this very
purpose, and we do not do that for fun, but because we actually have users
who want to compile software. Developers are explicitly one of the personas
for Fedora Workstation, and of course developers need to compile software!
But in the Free Software world, unlike the proprietary world, even non-
developer users may end up compiling software themselves for various
reasons.
In addition, build-time dependencies may also be useful for other purposes,
e.g., you may consider some LaTeX package to be a build-time-only tool used
during the building of your package's documentation, but I might actually
want to use that package in my scientific documents.
> *Requirement*: Build-time only dependencies for an alternative version may
> be excluded from the installable output artifacts. These excluded
> artifacts may be preserved by the build-system for other purposes.
As a result of the above, this should be an explicit non-goal.
> *Requirement*: All sources used for generating alternative versions,
> regardless of final visibility, must be available to the community for
> purposes of modification and reproducibility.
And this requirement actually conflicts with the preceding one. The best way
to make those sources available is as part of the SRPMs of the regular
repository, and the best way to ensure easy reproducibility is to actually
also ship the binaries there. If I want to modify or reproduce your build of
bar that depends on libfoo, why should I first have to track down the libfoo
sources from some hidden side repository and then rebuild libfoo from source
before I can actually compile bar, which is the only thing I actually wanted
to compile?
> Defining the public API
>
> Similarly, there are times when an application the packager cares about
> depends on another package that is required at runtime, but sufficiently
> complex that the packager would not want to maintain it for general use.
> (For example, an application that links to a complicated library but only
> uses a few functions.)
As already explained above, it is then standard practice in a community
distribution that somebody needs to bite the bullet and just maintain this
library with no strings attached. Comaintainership (where different
comaintainers are most familiar with different parts of the library) can be
a solution if there are suitable candidates, but even if there are none,
there needs to be at least one maintainer.
> In this case, we want there to be a standard mechanism for the packager to
> be able to indicate that some of the output artifacts are not supported
> for use outside this module. If they are needed by others, they should
> package it themselves and/or help maintain it in a shared place.
>
> *Requirement*: Packagers must be able to encode whether their output
> artifacts are intended for use by other projects or if they are
> effectively private to the alternative version. Packagers must also have a
> way of finding this information out so they understand what they can and
> cannot rely on as a dependency.
Such arbitrary restrictions of the "supported" interface are both a very
anti-social thing to do and a conflict waiting to happen. The latter because
(as already pointed out above) even "private" packages still share the
installation's global package name namespace and file system, so if every
module bundles their own copy of libfoo with a different "supported" subset,
those copies will conflict on end user systems.
In addition, Fedora does not provide commercial support, and as such,
deciding whether to "support" some functionality or not is pretty
meaningless in Fedora.
Hence, no, I do not agree that packagers should be enabled and allowed to
encode such arbitrary "support" restrictions in Fedora.
> Use-case-based installation
>
> Since the earliest days of Linux, the “package” has been the fundamental
> unit of installable software. If you want to have some functionality on
> the system, you need to learn the name of the individual packages that
> provide that functionality (not all of which are named obviously). As we
> build modules, one of the goals is to try to focus installation around
> use-cases rather than around upstream projects. A big piece of this is
> that we want to have a way to install a subset of a module that supports
> specific use-cases. A common example being “server” and “client” cases.
>
> *Requirement*: It must be possible to install a subset of artifacts from
> an alternative version. These installation groups should be easily
> discoverable.
>
> *Recommendation*: Installation groups should be named based on the
> use-case they are intended to solve. This will provide a better user
> experience.
This functionality is almost orthogonal to the parallel availability of
alternative versions. Yet, both are being mingled together in "modules".
We already have two existing mechanisms to encore super-package units: comps
groups and metapackages. Now modules add a third, incompatible way to
provide this same information, and one which is mixed with a choice of
version.
Why can enabling modules not just work like enabling repositories (i.e., it
selects which package versions you get if you install the packages, but does
not actually install anything) and the actual groups be kept in comps with
all the existing ones? I consider the fact that selecting a version of
something also forces a specific set of packages on me, which I may or may
not actually want or need, very confusing. E.g., I may want to enable a
specific stream of a module for git and only get git-core from it and not
whatever git* packages are considered part of the smallest "use case" by the
module maintainer. (In the worst case, there may only be unsuitable "use
cases", such as a "client" use case including some GNOME GUIs that I have no
use for on my KDE Plasma system, and a "server" use case that includes some
server packages that I have no use for either.)
> Lifecycle isolation
>
> Another of the major issues faced by Fedora is maintaining a release
> schedule when all of the components within it follow vastly differing
> schedules. There are two main aspects to this problem:
>
> - A major version of a popular piece of software is released just after
> a Fedora release, so it doesn’t land in Fedora for six months.
> - Some software does frequent major revisions (Django, Node.js, etc.)
> and swapping them out every six months for the latest one means that
> dependent projects are constantly needing to adapt to the new breakage
> or find alternative mechanisms for retaining the older, working version
> - Some software does not handle multiple-version upgrades (Nextcloud,
> for example). Attempting to go from version 15 to verison 19 requires
> first upgrading through 16, 17, and 18.
>
> *Requirement*: It must be possible for new alternative versions of
> software to become available to the Fedora Package Collection between
> release dates.
As already mentioned above, this is already possible in at least 3 different
ways (pushing the new version as an official update, creating a versioned
compatibility package, providing the alternate version in a Copr).
For leaf applications, compatibility packages are not really a good fit.
There, a non-parallel-installable stream selection (as provided by the
module streams) indeed makes more sense. But this was already possible
through Copr.
For frameworks such as Django and Node.js, the lack of parallel
installability is an issue, because it means that some applications will not
be coinstallable because they depend on different versions of the framework.
So modules are a poor fit for those. Compatibility packages (i.e., parallel-
installable packages with version-suffixed names) should be used instead. I
do not see why I should not be allowed to install two unrelated web
applications on the same web server just because they depend on different
versions of Django or Node.js.
> *Requirement*: It must be possible for alternative versions of software to
> go end-of-life during a Fedora release. This does not mean that the
> software must disappear from the repositories, merely that an assertion
> exists somewhere that after a certain date, the package will not receive
> updates.
I think that this is a very bad idea, because it is really confusing for
users, and I am not alone in thinking like that, because this idea was
actually already voted down by FESCo. (FESCo decreed that modules are only
allowed to go EOL with a Fedora release EOL.)
> *Requirement*: For alternative versions whose lifecycle will continue
> through at least part of the next Fedora release, it must be possible to
> upgrade from one release to the next and remain with the fully-compatible
> version.
The question, though, is whether that really needs to be the default action
or whether it would not make more sense to upgrade to the new default
version by default and require an explicit downgrade to retain the old
version (and all the more so if the module maintainer is not even going to
maintain that old module stream for the lifetime of the new Fedora release).
I would argue for the latter.
> Third-party additions
>
> Some third-party add-on repositories (particularly EPEL) have been limited
> in the past by relying on the system copies of packages in the base
> distribution of the release. In the particular case of EPEL, little can be
> done to upgrade these system copies. In order to be able to package much
> of the available FOSS software out there, it may be necessary to override
> some of the content shipped in the base system with packages known to work
> properly.
>
> *Requirement*: It must be possible for third party repositories to create
> alternative versions that override base distribution content at the user’s
> explicit choice.
>
> *Requirement*: It must be possible for third party repositories to create
> alternative versions of software that exist in the base distribution.
It is not clear to me what the difference between those two requirements
would be. They sound like two different wordings for the same thing to me.
Either way, this is all about bypassing the EPEL policy of not replacing
core RHEL packages. But that policy was enacted for a reason. Doing this for
non-leaf packages can be quite dangerous, potentially breaking both software
included with RHEL and third-party software depending on the original RHEL
version of the library. As long as the replacement only happens at a user's
explicit request, the potential for damage is limited, but the point of
replacing a non-leaf package with a different version is usually to enable
some application depending on that version, so that application will then
drag in the non-default version of the non-leaf package (with the potential
for breakage that that entails).
E.g., the KDE packager for EPEL 8 wants to upgrade Qt to a new version
through a module. Installing the Plasma module will then also upgrade the
system Qt. One can usually get away with that due to Qt's backwards
compatibility, but some packages depend on private APIs or on undocumented
behaviors and will thus not work with a new Qt. If those packages are not
all rebuilt in the module upgrading Qt, they will break. So I am not
convinced that upgrading core RHEL packages through modules is any safer
than just upgrading them through the main repository. If it is decided that
upgrading Qt and rebuilding those packages known to need a rebuild is safe
(after all, this is routinely done in Fedora, though of course Fedora is
less concerned about third-party binary-only software than RHEL), it should
be done EPEL-wide (or ideally in RHEL proper, but that is out of the
community's control), whereas if it is ruled to be unsafe, it should just
not be done anywhere.
For Fedora proper (i.e., outside of EPEL), this requirement is not
applicable at all, because we are speaking about a specific EPEL policy.
> Reduce duplication in packaging work
>
> There is plenty of software out in the wild that maintains compatibility
> over time and is therefore useful to carry in multiple releases of Fedora.
> With traditional packaging, this means carrying and building separate
> branches of the packages for each release of Fedora. In the case of
> software “stacks” which are tightly bound, this means also manually
> building each of its dependencies in each release of Fedora.
>
> *Requirement*: It must be possible to build multiple component software
> packages in the same build process.
>
> *Requirement*: It must be possible for the packager to specify the order
> in which packages must be built (and to indicate which ones can be built
> in parallel).
>
> *Requirement*: It must be possible to be build for all supported platforms
> using the same specification and with a single build command.
These make sense as nice-to-haves (I would not call them requirements), but
they all really call for better packaging automation, not Modularity. We do
not need a new delivery mechanism (modules) to achieve that. This can all be
done manually (using git fast-forwards, chain builds, buildroot overrides or
side tags, etc.) now, what is missing is a way to get it done in one command
and directly from git master (to avoid having to fast-forward the branches
each time). But that mechanism can be purely a build-time thing and need not
affect the way the packages are delivered to our users at all.
> Non-Goals
>
> Parallel installability
>
> As mentioned in the Background section, the goals of Modularity are
> specifically to *not* implement parallel-installability. We recommend
> instead that users should rely on other mechanisms such as virtualization
> or containerization to accomplish this.
As explained above, that may work for enterprise RHEL customers, but is not
a workable approach in the Fedora world.
> If parallel-installation is unavoidable, then Modularity is not the
> correct tool for this job.
Agreed, but this essentially disqualifies Modularity for any non-leaf
package, and even for some rare leaf packages (e.g., where old versions are
needed to read old documents).
> Arbitrary stream switching
>
> Module streams are intended to be compatible update streams. That means
> they must follow the same rules regarding RPM package-level updates within
> the stream. By definition, two streams of the same module exist because
> upgrades (or downgrades or cross-grades…) are not capable of being done in
> a safe, automated fashion.
>
> That does not mean that stream switching should be impossible, but it does
> mean that we will not build any tools intended to handle such switching in
> a generic manner. Stream switches should be handled on a module-by-module
> basis and detailed instructions and/or tools written for each such case.
Sure, a new module stream is by definition incompatible in some way, or it
would not make sense to create a different stream to begin with, I follow
you so far. But in Fedora, we already have well-defined points at which such
incompatibilites may occur: Fedora releases. (I concede that this is more
fluid in the RHEL world where, due to their extremely long lifetime, the
major releases get "update releases" that try to be fully compatible, but
occasionally contain somewhat incompatible changes. But here, we are talking
about Fedora only.) Therefore, I think that it is entirely reasonable for
Fedora release upgrades to default to upgrading to the new release's new
default version of the packages and modules. That would of course only be
the default, it would always be possible to downgrade to a specific module
stream, or ideally to explicitly exclude it from being upgraded during the
upgrade to begin with. But I do not see why the distro upgrade not upgrading
the module needs to be the default behavior.
Kevin Kofler
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
