----- Original Message -----
> From: "Kevin Kofler" <kevin.kofler@xxxxxxxxx>
> To: devel@xxxxxxxxxxxxxxxxxxxxxxx
> Sent: Tuesday, November 5, 2019 7:48:47 PM
> Subject: Re: Modularity: The Official Complaint Thread
>
> Alex Scheel wrote:
> > Special care needs to be taken to make sure we discuss what happens
> > when a _module maintainer_ wants to switch the module stream of one
> > of its dependencies, especially a dependency the user never
> > specifically selected a stream for. That should be an allowed and fully
> > supported use case.
> >
> > This was the pki-core<->idm module fiasco that was never resolved by
> > DNF/Modularity.
> >
> > I'd post the bug number but the bug isn't public.
> >
> >
> > So perhaps split this into:
> >
> > 1. How does a _user_ change module streams during upgrade?
> > 2. How does the _maintainer_ change module streams of a dependent
> > module? (module a -dep-> module b -- change stream of b from 1 to 2)
> >
> >
> > IMO, without a resolution in Fedora we'll never get one in RHEL.
>
> Indeed, in Fedora, it is quite plausible for a package to be ported to a new
> major version of a library in an update. (In RHEL, it actually also is, but
> for different reasons, i.e., its extremely long lifetime.)
>
> But this shows how modules are the wrong answer for non-leaf packages. What
> happens if one of the packages you had installed wants to bump the module
> stream of its dependency and another one doesn't? Then suddenly, your system
> cannot be updated anymore, because the packages that previously coexisted
> just fine now irremediably conflict.
>
> (In fact, this is just a particularly nasty special case of the more general
> design flaw of Modularity that Stephen Gallagher has unfortunately forgotten
> in his list in the original post, the version conflicts caused by versioned
> dependencies without parallel installability. The special case is that the
> conflicts can also be introduced on updates to previously non-conflicting
> packages.)
>
> Providing incompatible versions of non-leaf packages MUST be done in a
> parallel-installable way. There is no other way out.
Since this is all public, for the rest of the audience:
- There are three modules in question here, in core RHEL:
1. pki-deps, an artifact of the original, broken attempt at Modularity
in RHEL, when builds would take eons. This contains a bunch of
dependencies of Dogtag.
2. pki-core, the core packages of Dogtag:
- JSS
- Tomcatjss
- ldap-jdk
- Dogtag PKI
3. The IPA module and its "DL1" stream.
You can pull these three modules today on RHEL and try them out if
you want.
- The dependency tree is thus:
IPA:DL1 -> pki-core:10.6 -> pki-deps:10.6
- What we had wished to do was provide two sets of module streams:
pki-core @ 10.6 -- Dogtag at 10.6 version (in RHEL 8.0 only)
pki-core @ 10.7 -- Dogtag at 10.7 version (in RHEL 8.1 only).
Anyone who was using pki-core by itself could stay on 10.6 if they
so desired (by staying on RHEL 8.0) or upgrade to 10.7 (by upgrading
to RHEL 8.1). This allowed us, in the future, to support multiple
versions on the same RHEL version if there were breaking changes
between them.
Since IPA is a monolith and wanted features in 10.7, it was going to
switch to 10.7 in the new RHEL 8.1 release (out today!) because it
wanted some things we introduced there.
- Instead, since switching branches as described above wasn't supported,
we ended up shipping our 10.7 version release in the (now incorrectly
named) 10.6 branch. This means, on the whole, the pki-core module isn't
netting us any value. It functions just like an ursine collection of
packages with only a single version stream available.
So to answer your question:
> What happens if one of the packages you had installed wants to bump the
> module stream of its dependency and another one doesn't?
This was RHEL. We (the Dogtag / RHCS team at Red Hat) had full control
over our package. We could choose to ship whatever version we want. That
meant that we have to _work_ with other teams (in this case, exactly one:
IPA) and ensure what we wanted to ship wouldn't break anything. Part of
that was making sure we can switch streams. It turns out that, while we
wanted to and got sign-off from IPA, the issue was a fundamental limitation
in DNF. In RHEL, we know there is only one such dependent, IPA, and we can
deal with that.
In short, inside RHEL, we can take this as risk and control for it. In
Fedora, it is harder and would require (packaging/FESCO/Modularity/...)
policy to enforce. But it really _isn't_ a hard thing to do: limit it to
Rawhide, require changes to be announced and coordinated (like SONAME bumps).
Plus, inside RHEL, we could make reasonable support assumptions like, say,
running an IPA server on the same system as $MythicalOtherUserOfDogtag wasn't
supported. I'd argue we could (and should!) do the same in Fedora. In
practice, that usually happens by maintainers ignoring bugs reported that
they don't like.
(As an aside, while Dogtag isn't parallel-installable from a versioning
perspective, it is perfectly valid to have multiple Dogtag instances on
the same host. We do maintain backwards compatibility and provide
automatic migrations to newer versions, when possible. This means that
there really is no reason not to upgrade. And modularity's lack of upgrade
and module stream switches don't really affect us.)
---
I don't personally think this is any more of a showstopper than, say,
default module streams shadowing content from ursine packages. Or say,
building huge BuildRoot-only modules. Or say, duplicating packaging
efforts between multiple modules.
As a community, on the whole, our approach to the other issues has been
best summarized as:
"eh"
We'll deal with it when it is actually a problem. I think we can take
the same approach here.
Then again, we've avoided this entire mess by staying ursine in Fedora.
We've tried to be good stewards for the community by maintaining as many
ursine packages as we could through the Stewardship SIG. The net results
of the SIG have been a reduction from 70% out of date to 31%, to date.
---
Lastly, I think Modularity looks at the problem of packaging from the
wrong perspective. It encourages monoliths and heros, instead of well,
smaller (more modular!) packages and community efforts. Maintainers are
failable. (Most) maintainers don't have rights over packages that aren't
theirs. And software needs to move _forward_ not stay stagnate. And the
FTBFS process helps with some of this, removing packages that won't
continue building.
Concretely, if you aren't the maintainer for a package, I don't think you
have a fundamental right to keep it at some older version "just because",
if the _actual_ maintainer wishes to bump the package version. So in general,
it _should_ be allowed to bump the module stream. And perhaps DNF should
resolve packages to highest NVR across all enabled (or depended on streams).
But this is complicated because there was no actual constraints placed on
streams, and modularity introduces a whole different dimension of complexity.
So I think, with reasonable policies protecting stable releases from module
stream switches, let them happen. The community needs to work together. The
community will work it out as a group.
Instead, we spend too much time duplicating each other's work...
... and arguing on mailing lists. :)
My 2c.
- Alex
>
> Kevin Kofler
> _______________________________________________
> devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
>
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
