Stephen Gallagher wrote:
> On Wed, Oct 16, 2019 at 7:58 PM Kevin Kofler wrote:
>> So completely disable all module support in DNF by default with some
>> global flag (make all the module code conditional under some new
>> enable_modules flag and default the flag to enable_modules = 0), then it
>> will treat the packages as normal packages and you only have to provide a
>> higher EVR. All this module processing should only happen if the user
>> explicitly enables it.
>
> Given that many of the modules in the distribution currently are there
> specifically to provide newer, less-stable versions of the versions in
> the non-modular/default stream, this would be fairly disastrous. For
> example, Node.js 10.x is the default stream in Fedora 30 because it's
> the LTS branch. It also provides a non-default stream for 8.x (the
> previous LTS branch that a lot of applications still rely on) and
> 12.x, which will be the next LTS branch in November, but is not
> guaranteed stable yet. With the approach you're describing, everyone
> would be forcibly updated to the unstable 12.x release.
>
> The only way we could assure ourselves that this wouldn't happen would
> be to do another mass-rebuild, bumping the epoch of every package that
> exists in a module. That's a lot of work.
So it looks like I did not describe clearly enough what my proposed
enable_modules=0 flag would do. ("Disable all module code" was apparently
too vague.)
How I think it should work would be:
* For repositories, it completely ignores modular metadata and processes
only the non-modular parts of the repository metadata. Therefore, it does
not see the Node.js 12.x stream at all. It only sees whatever Node.js is
in the non-modular repository. If there are currently only modular
versions, then it sees none at all. But with the proposal to require a
non-modular default version, it would then see that version, which would
likely be Node.js 10.x. Nobody would get forcefully upgraded from 10.x to
12.x.
* For installed modules, it completely ignores them, acting as if the
database of installed modules were empty. (It just does not read that
database at all.)
* For installed packages, it treats them all as non-modular. Sure, packages
originally installed from a module have weird EVRs encoding module
metadata, but otherwise they get processed exactly like a non-modular
package. So the default repository only has to provide a newer EVR to
upgrade the package. That should address upgrades from 8.x or 10.x to the
new default 10.x. If the user had previously installed 12.x, they will
only get downgraded if they distro-sync or if package-level dependency
issues with the F30 build of 12.x on F31 force the downgrade.
I hope that clears it up.
>> A slightly more elaborate, but slightly harder to implement, approach
>> would be to let DNF simply disable modules that are enabled locally but
>> no longer available in the repositories, together with disabling the
>> fedora-modular and updates-modular repositories by default.
>
> And again, this only works if every packager who has spent time
> creating a module with a default stream takes their content and shoves
> it back into the non-modular repository. Which in some cases they
> probably cannot do, because they have build-dependencies that are in
> conflict. This is a highly non-trivial process and it would need to be
> done individually for every single package. That's far more
> packager-hostile than fixing the default stream/buildroot problem and
> the upgrade path problem.
You are overestimating by far the effort required to demodularize the
handful packages that are currently module-only. The evidence Fabio
Valentini has gathered so far shows that actually very few packages would be
affected and they would not be too hard to fix. And Miro has also offered
help with fixing affected packages.
All in all, it would require fixing a handful packages once and for all
instead of implementing workarounds affecting the entire distribution and
its thousands of users.
>> And the case of demodularizing packages has to be addressed sooner or
>> later anyway, so better address it sooner rather than later, before more
>> and more damage is done by maintainers moving packages to module-only
>> without a way back.
>
> I've already acknowledged upthread that demodularizing packages is a
> problem we need to solve. It's being worked on, but it's a lot harder
> than you think, because we have failsafe code implemented in libdnf to
> prevent *accidental* demodularization that's in conflict with this
> desire.
And exactly that code needs to go, at least in Fedora. I think having a way
to migrate away from modules is the common case to prioritize here.
That said, it shall be pointed out that, if the proposal to demodularize all
default versions of packages gets implemented, we only need a *short term*
solution for demodularization in DNF. After 2 releases, we have no default
streams left (and they will never come back by policy) and we can expect
users to have upgraded through a release with no default streams (given that
we do not support upgrading directly to n+3), so at that point DNF can
revert to the "safe" behavior (preventing accidental demodularization) by
default.
So the proposal to demodularize everything could actually make this problem
easier to solve.
> Also, this paragraph was needlessly antagonistic: moving packages to
> modules is not "damage".
Sorry for that. The reason I called it "damage" being done is that there is
currently no supported way back (as you pointed out yourself) and that it
moves us away from the state I (and others, e.g., Miro) believe we should
reach (where there are no module-only packages anymore).
I consider this approach of making a controversial and experimental change
with no contingency plan, then using that absence of a contingency plan as
an argument to not only stick to the change at all costs, but even go
further with it, entirely unacceptable. (We call it the "creating facts"
("Fakten schaffen") tactic in the German-speaking parts of the world. It is
an effective way to bypass discussion and democratic participation.)
> Sure, but we are nowhere near a cliff. As I just posted in the Change
> Proposal thread, there are three problems we need to solve, two of
> which we already have solutions designed for and one (this thread)
> that we are trying to finalize. That's far from "standing in front of
> a cliff".
The problems you listed are not the only ones. There are the version
conflicts between non-default streams that can be dragged in by default
streams. There is the buildroot-only content, which breaks our self-hosting
promise and makes it harder to compile additional software. There is the
issue that modules built for an older distribution cannot be kept on a newer
distribution version, which is a regression from ursine packages that (e.g.,
if they get dropped from the distribution) you can keep as long as they
don't depend on an outdated soname, which can be decades. There is the risk
that your more and more complex solutions introduce new issues, e.g., bugs
and undesirable behavior in DNF, that may or may not be easy to fix. (The
more complexity you introduce, the harder it gets for DNF to behave the way
the user expects.)
And this needs to be brought into relation with the benefits of using
default streams instead of non-modular default versions, which as far as I
can tell are essentially nonexistent. (Sure, they might be an easier upgrade
path for things that are already default streams now, but this is self-
referential. We should not stick to default streams forever only because of
a historical decision. And actually, the upgrade path from default stream to
default stream is also not working yet, which is why we have this thread at
all.)
> Please understand that "I don't understand how this works" is not the
> same thing as "This doesn't work".
I understand very well how this doesn't work. :-)
> So, literally every word of this is wrong. The negative feedback is
> not "overwhelming". It is approximately four noisy individuals, all of
> whom have expressed zero interest in understanding the actual
> situation that they are trying to "fix" by endlessly insulting the
> people working on the problem. Demoralizing the people who can dig us
> out of this situation is an unwise strategy.
LOL, filing off mailing list consensus as "four noisy individuals". This is
getting really ridiculous! Why am I even arguing with somebody who clearly
does not want to listen?
And you are the people who brought us into this situation to begin with.
Default streams should never have been allowed without:
1. an upgrade path from default stream to default stream, AND
2. a contingency plan, i.e., an upgrade path from default stream to
non-modular default version
The fact that they were implemented without EITHER of these (when actually
BOTH are needed) was extremely short-sighted.
So bringing yourselves up now as "the people who can dig us out of this
situation" feels to me feels like intentionally making a patient sick so you
can "cure" them.
> Also, we're not offering "partial workarounds" (excepting some
> acknowledged hackery to avoid blocking F31). All of the proposals I
> have been discussing in this thread are for real design adjustments
> for long-term benefit.
I call them "partial" because they are do not address all the issues we are
having with Modularity. See my list a few paragraphs above. At least the
version conflict issue is not fixable at all without a complete redesign of
Modularity that would be incompatible with the current implementation and
would probably also violate the FHS (because that is the only way to achieve
universal parallel-installability without per-package workarounds).
And I call them "workarounds" because they add more and more complexity when
there would be a simple fix: just don't use default streams anymore. That
change might also need short-term workarounds for the upgrade path, but
those can all be dropped once the default streams are gone.
> And while they add some additional complexity on the *infrastructure*, a
> primary goal is to not make the users or packagers' lives harder.
Yet that is exactly what default streams are doing.
> We *know* that the default stream/buildroot issue is failing to hit this
> goal and the solution is known, implemented upstream and could be deployed
> by the end of the week if FESCo gives its approval.
The buildroot workaround does not address any of the issues the end users
are hitting. You are proposing a different workaround for the upgrade path
issue for end users, workaround which as far as I can tell is not
implemented yet, which will not address the other issues, and which will
still lead to a more complex feel than the simple fix of no longer using
default streams.
I do not understand why you are so strongly against the "no default streams"
rule. It would not stop any of the work you are doing on Modularity. You
would still be able to ship all your module streams, depend on any other
stream of any other module (even if some combinations of streams then
conflict), and continue your work as before. The only difference is that
instead of making a stream the default, you would merge it into the release
branch for the release and build it as a normal "ursine" package. But that
does not conflict with any of the Modularity work you are doing.
> No, you've decided on the outcome you want to see and have invented a
> path to get there that doesn't align with the realities of the
> present.
Funnily, that is exactly how I would describe what has happened with
Modularity so far.
Kevin Kofler
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
