On Wed, Dec 19, 2018 at 09:54:07AM +0100, Michal Novotny wrote: > Hello Fedora devels! > > TL;DR: let's use rpm DistTag (DistTag, not the %dist macro) to denote > stream names > > As we all know, modularity was created to provide > parallel-availability, that is providing the same package in its > different major versions from the same rpm repository. > > The first question to ask is: "Why can't we just place two major > versions of a package alongside in the repo?" > > That's possible but there are two problems with this. The first one is > that when you type > > $ sudo dnf install git > > dnf needs to determine what major version of git it should install. > User should be able to provide the information explicitly on > command-line, that is: > > $ sudo dnf install git-1 > > or > > $ sudo dnf install git-2 > > but doing that always for each package you install could be quite demanding. > > The second problem is that once you have a git package of a certain > major version installed, you want to keep it on that version when > running an ordinary update command: > > $ sudo dnf update git > > dnf shouldn't be tempted to install git-2 (e.g. git-2.17.2-2) but it > should instead update git-1 on the system to the latest available > minor version if git-1 is the major version of git currently > installed. > > Modularity solves the second problem by "streams" and the first > problem by a "default stream". > > That's okay but I would argue there is much more simple solution to > those two problems, which reuses already known terminology and > technology. > > What can be done is to put the information into which stream a package > belongs into the package spec file. That means, a packager will not > need to edit some additional files to enable parallel-availability. > > I would argue that if every single package in Fedora used semantic > versioning, then we wouldn't need streams at all. dnf could simply > have a config option that would assure staying on the same major > version during updates unless told otherwise. > > The concept of "stream" is only needed if there are upstreams/packages > that do not exactly conform to semantic versioning where change in the > first number means API breakage. I suspect there are quite a lot of > them still and maybe, in some cases, we would like to have streams > that stand for something else than a major version? E.g. devel that > stores always the latest upstream state and is updated automatically. > > For that reason, I assume the concept of stream might be useful. > > But the question is how to provide a stream name through rpm? > > I was looking at > https://github.com/rpm-software-management/rpm/blob/master/lib/rpmtag.h > and found there is DistTag rpm tag that you can put into a spec file > and that seems to be unused these days. > > According to what I have found in a comment in /usr/lib/rpm/macros on this tag: > > # Configurable distribution tag, same as DistTag: tag in a specfile. > # The tag will be used to supply reliable information to tools like > # rpmfind. > # > #%disttag > > Fedora packages do not have it set: > > E.g.: > > $ rpm -q --queryformat '%{DISTTAG}\n' prunerepo-1.13-1.fc28.noarch.rpm > (none) > > and I don't think we need to care that much about rpmfind at least as > far as DistTag is the concern. > > So it could be used to provide a "stream" name. I mean we don't really > need to call it stream either. What we essentially want is to group > packages of the same name by their major version or in some case by a > descriptive string like "devel". > > Once we have this grouping provided for dnf, we can add an option like: > > updates_maintain_disttag=true/false > > which will make dnf updates stable if true and if packagers keep their > DistTags set to package major versions. > > Another thing is that DistTag value can be automatically derived from > DistGit branch name. Meaning that if you use arbitrary branching like > e.g. nodejs uses (https://src.fedoraproject.org/rpms/nodejs/branches) > where branch names correspond to major versions, then with some kind > of auto-generation in place you can be just-about done only by > creating a branch with a proper name. > > Funny thing is that this would work even if used with distro branches > because in distro branches, packagers should not update majors per > packaging guidelines (only maybe for leaf nodes with FESCo exception > it might be possible) so even though such stream (e.g. f29) would be a > mix of many different packages, those packages would maintain the same > major version during the stream active existence. > > The second problem was to make > > $ sudo dnf update git > > work if there are e.g. two DistTags ("streams") 1 and 2. > > Do you pick git-1 from DistTag 1 or git-2 from DistTag 2? > > I would leave the decision what version to pick on a dnf config > option, which could be e.g.: > > default_disttag_select_strategy=lowest_compatible > > where sorting of DistTags follows the rules used by sort linux command > in en_US.UTF-8 locale: > > $ sort > 0 > 1.1 > devel > 3 > 1.11 > > 0 > 1.1 > 1.11 > 3 > devel > > This means git-1 in its highest minor version is picked but only if > its compatible with already installed packages in the system. If not > compatbile, git-2 (having the next lowest DistTag) is examined for > compatibility. If not compatible, an error is raised unless there is a > git version with DistTag 3. > > Is this what we want? I would argue, yes. We can have the lowest > compatible versions as the most stable common ground and if a major > version is no longer supported, we can remove its DistGit branch or > rename it to unsupported-<major>. That way the package could still be > temporarily available for explicit install but no longer a default. > > What becomes the default DistTag doesn't depend only on what branches > you have in your DistGit repo but also on what packages you actually > build and release for a given distro version. We could have e.g. > Fedora Devel where only devel branches go or just simply Fedora-30 > with up-to-date defaults. A Fedora "Base" rolling distro containing > all the active DistTags for each package would be in this sense the > most conservative one. Even though it would contain the most recent > versions of every package, they wouldn't be installed by default until > they become the lowest major version (until they have the lowest > DistTag, more precisely) or until the lower DistTags are incompatible > with what a user has installed. But of course, users could always jump > up from the base for any package by an explicit-version install (e.g. > $ sudo dnf install git-2), which could potentially update lots of > packages to satisfy the Requires. > > Update system could provide some nice interface for this that would > visually group packages by their DistTags and group the DistTags by > the actual Fedora releases. It could also check that DistTags are not > "downgraded" for a release by mistake. > > ---- > > I believe this suggestion provides the same essential features as the > current modularity implementation while keeping things simple for both > users and packagers. > > User shouldn't notice anything except that there are suddenly multiple > major versions of a package available and that it works as they > expect. > > Packagers only need to take care of choosing a good branch name and > sending built packages into desired releases by using Bodhi. > > Modules simply become package repos again and streams are simply their branches. > > I would then also argue that other features of modularity can be > provided on purely rpm level too: > > profiles: separate rpm per profile - e.g. vim-enhanced vs. vim-minimal > > possibility to include multiple packages within a stream - rpm > Requiring other rpms and providing a binding config for them. > Optional. > > clime So, first, I'd like to thank you for taking the time to think about this. It's certainly interesting and tries to solve similar problems from a completely different angle. Your proposal focuses on distribution packages rather than on independent applications. I'd argue it introduces even more packaging complexity (all packages would need to be modified to depend on specific "streams" of other packages) than grouping unmodified packages into modules. I also worry that with such a fine-grained parallel availability, there would be very few possible combinations of packages that could be effectively installed on a single system. Not to mention QA efforts. -- That aside, a couple points touching some aspects of your proposal that you might interesting: * DistTag isn't actually unused. The current modularity implementation uses it to label MBS-procuded RPMs. The tag is also used by other distributions and we're currently introducing a new header to avoid the conflict. https://pagure.io/modularity/issue/113 * Soon, users will be able to choose whether they follow the defaults for each individual module or stay on the stream they initially picked/enabled. https://pagure.io/modularity/issue/108 * We do not compare stream names in any way, they're just strings. Streams don't have to be just major versions of upstream software, they can indeed be anything the packagers want them to be -- experimental builds, builds with different compiler flags, builds with or without specific dependencies, and so on. Modules can build packages from the same sources in different buildroots or with different macros easily to enable this. P
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
