Hi! I'm trying to get https://fedoraproject.org/wiki/Changes/Unify_bin_and_sbin implemented. It was approved for F40, but only a few days before the mass rebuild, so there wasn't time to do much, so it was retargeted to F41. We now have some time before the F41 mass rebuild, so I want to push all the changes required in various packages so that we have time to work out any kinks before the mass rebuild commences. When I started looking at individual packages, I discovered that we have an old rule that packages are SUPPOSED to use "historical paths" to list their files. This rule was introduced to make the transition easier when UsrMove was implemented for F17. For example, mount.nfs was historically installed as /sbin/mount.nfs, so nfs-utils must use %files:/sbin/mount.nfs, even though /sbin is a symlink to /usr/sbin, meaning that the real path after installation is /usr/sbin/mount.nfs. And packages using a file path dependency on mount.nfs must use /sbin/mount.nfs too. To implement this rule, packages must use %buildroot/sbin during installation and use literal "/sbin/" in the files section. This idea made sense at the time, but now it seems overcomplicated and unecessary. It requires additional work from packagers who create the packages that provide the files, but also additional work from packagers that want to refer to those files. In fact, I only found three packages that implement the rule correctly: nfs-utils, glibc, and ocfs2-tools. So step 0. is to drop the packaging rule: https://pagure.io/packaging-committee/pull-request/1355 As to the actual sbin merge: Currently, we have %_bindir==/usr/bin, %_sbindir==/usr/sbin, and /sbin->/usr/sbin, /bin->/usr/sbin. In the end state, we will have %_bindir==%_sbindir==/usr/bin, and /bin,/sbin,/usr/sbin -> /usr/bin. This end state is simple: _any_ path will works for any binary. It's the transition, as we rebuild packages with the new definition, that requires some careful handling. After experimenting with a few different ways to handle this, the following approach seems to work best: 1. rpm is patched to provide %_sbindir==/usr/bin. (This affects what paths packages will list after being rebuilt.) 2. filesystem is patched to not contain /usr/sbin in %files, but instead symlink /usr/sbin -> bin in %pretrans. On fresh installations, this will succeed, so the merge is in place. On upgrades, this will fail, meaning that /usr/sbin remains a dir. There is also a %posttrans scriptlet to attempt a merge on upgrades, more about this below. In particular, since buildroots are created afresh for each build, package builds will get merged-sbin. 3. We need to handle the case where package foo had /usr/sbin/foo, but this file will be in /usr/bin/foo after rebuild. After the merge is done, /usr/sbin/foo and /usr/bin/foo will point to the same location, no problem, but during the transition, users or scripts might call /usr/sbin/foo. To make this work, filesystem rpm gets a scriptlet that will create symlinks from /usr/sbin to /usr/bin, for any files which were installed by packages in /usr/sbin. (A list was obtained using repoquery.) Initially, I wanted to put those scriptlets in individual packages, but that turns out to be a lot of duplicate work. Some packages have multiple subpackages with files (currently) in /usr/sbin, so we'd end up with a lot of churn. Doing it once in filesystem turns out to be fairly easy. 4. We also need to handle the case where other package bar has Requires:/usr/sbin/foo. Once foo has been rebuilt, it has only an automatic provides for /usr/bin/foo. We could adjust bar for the new path, but then bar would not be installable on old systems. Instead, a compat virtual Provides:/usr/sbin/foo is added to foo. We know that either /usr/sbin will be a symlink, or that filesystem will create the /usr/sbin/foo symlink. We need to ensure that the filesystem package that is installed is actually new enough so that the Provides is not a lie. filesystem.rpm gets a virtual Provides:filesystem(unmerged-sbin-symlinks)=1 which is used as Requires:filesystem(unmerged-sbin-symlinks) by foo. (An explicit Provides/Requires allows us to adjust or rescind the mechanism in the future.) 5. After this preparatory work is done, we can rebuild various packages with updated rpm and filesystem. Packages which don't use %_sbindir generally don't care. Some packages which use %_sbindir need small adjustments. There are some common failure modes: a. 'mv %buildroot/%_bindir/foo %buildroot/%_sbindir/' b. 'ln -s ../bin/foo %buildroot/%_sbindir/foo' c. Listing both %_bindir/foo and %_sbindir/foo in %files d. 'ln -sf ../bin/foo %buildroot/%_sbindir/foo.alt' e. 'ln -sf ../sbin/foo %buildroot/%_bindir/foo.alt' To make builds work both before and after the merge, a-c should be conditionalized on '%if "%{_sbindir}" != "%{_bindir}"'. d-e are fixed by using 'ln -sf --relative %{buildroot}/%_bindir/foo %buildroot/%_sbindir/foo.alt' or 'ln -sf --relative %{buildroot}/%_sbindir/foo %buildroot/%_bindir/foo.alt' I submitted / will submit a bunch of pull requests for various packages, but not all, so I'm describing this here so that packagers can use this as a hint. 6. As we rebuild packages in merged buildroots, and those packages are deployed on user systems, in completely new installations, /usr/sbin will be created as a symlink. On upgraded installations, /usr/sbin is a directory and will contain a mix of files and symlinks. filesystem.rpm also has a scriptlet to check if /usr/sbin contains symlinks only, and once that's true, it will nuke /usr/sbin and replace it by a symlink to /usr/bin. This will finalize the merge on upgraded systems. The plan: I. Merge https://pagure.io/packaging-committee/pull-request/1355. This isn't stricly necessary, but makes things simpler. II. I've been doing test rebuilds of packages and filing pull requests as described in 1–4. above. Those should be merged before the other things, changes are conditionalized on '"%{_sbindir}"=="%{_bindir}"'. Some pull requests have already been merged. If you get a pull request, please review and/or merge. https://src.fedoraproject.org/rpms/rpm/pull-request/56 https://src.fedoraproject.org/rpms/filesystem/pull-request/11 https://src.fedoraproject.org/rpms/chkconfig/pull-request/13 https://src.fedoraproject.org/rpms/coreutils/pull-request/15 https://src.fedoraproject.org/rpms/cyrus-sasl/pull-request/11 MERGED https://src.fedoraproject.org/rpms/dmidecode/pull-request/8 https://src.fedoraproject.org/rpms/exim/pull-request/16 MERGED https://src.fedoraproject.org/rpms/exim/pull-request/17 https://src.fedoraproject.org/rpms/glibc/pull-request/91 https://src.fedoraproject.org/rpms/kmod/pull-request/12 https://src.fedoraproject.org/rpms/libcap/pull-request/29 MERGED https://src.fedoraproject.org/rpms/nfs-utils/pull-request/15 https://src.fedoraproject.org/rpms/ocfs2-tools/pull-request/2 https://src.fedoraproject.org/rpms/opensmtpd/pull-request/1 https://src.fedoraproject.org/rpms/policycoreutils/pull-request/42 https://src.fedoraproject.org/rpms/procps-ng/pull-request/7 https://src.fedoraproject.org/rpms/rpcbind/pull-request/4 https://src.fedoraproject.org/rpms/shadow-utils/pull-request/22 https://src.fedoraproject.org/rpms/systemd/pull-request/131 https://src.fedoraproject.org/rpms/util-linux/pull-request/17 III. I have two coprs: https://copr.fedorainfracloud.org/coprs/zbyszek/merged-sbin/builds/ https://copr.fedorainfracloud.org/coprs/zbyszek/unmerged-sbin/builds/ The first one has rpm and filesystem with the changes, so the builds done there get %{_sbindir}==%{_bindir}. Other packages are rebuilt with the patches from pull requests. The second has old rpm and filesystem, and it serves as a check that the same srpms as from the first copr still build fine without the merge. It should be possible to install the packages from the first copr onto a system and either end up with a system with some links and files in /usr/sbin, or /usr/sbin being a symlink. I still need to do more builds in the coprs. If it all turns out to work as expected, I think we're ready to execute the merge. IV. After the pull requests for rpm and filesystem are accepted, I'll build those in a side tag so that they appear in the buildroots at the same time. (Things would be even more confusing with just one of those.) [rpm, filesystem] There is a bunch of packages which do 5c in the list above. Those will FTI until they have been rebuilt. I'll build those in the same side tag so that the buildroot is not broken and we don't get cascading build failures. [rpcbind, systemd-udev, policycoreutils] Once that's done, the side tag can be merged and packages will see the new buildroot layout as they are rebuilt. V. Other packages will need to rebuild to change the layout. I think it'll make sense to rebuild any packages which have had patches applied. There is no "flag day", but it'd be great if we rebuild most packages with files in /usr/sbin, so that we can actually test if the full transition works as expected. VI. A special case of V. is packages using usermode helper with files in /usr/bin and /usr/sbin. Here, each package is different and input from maitainers will be needed. As long as those packages are not rebuilt, they will continue to function in the old state, but as point they'll need to be fixed or retired. I looked into setuptool, and AFAICT, the package is absolete and doesn't actually work. A candidate for retirement. I'm sure that I missed some corner cases. In case something breaks, let me know, I'll try to fix or help to fix. I want to finish working on rpm, filesystem, and other packages soon (hopefully this week), and merge everything about a week later. Zbyszek -- _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
