On 4/1/21 12:45 AM, Miro Hrončok wrote:
On 31. 03. 21 21:52, Ben Cotton wrote:
* Strict checking for unpackaged content in builds
> ...
* Many existing packages will fail to build due to the stricter
buildroot content checking. Fixing this in the packaging is always
backwards compatible. We could temporarily set
`%_unpackaged_files_terminate_build 0` in rawhide to alleviate initial
impact if necessary.
This is my main concern with this update.
tl;dr If you %exclude something and there is no other subpackage to own
the files, the build fails:
This fails:
%install
...
touch %{buildroot}/foo %{buildroot}/bar
%files
/
%exclude /foo
This still succeeds:
%files
/
%exclude /foo
%files foo
/foo
Many packages do the former in Fedora for various different reasons,
namely to, well... exclude files from the package (and not ship them at
all). Sometimes a `rm` in %install can be used instead. Sometimes not,
because the files are needed in the %{buildroot} for %check but not
needed to be shipped.
The bottom line is that the buildroot should not contain anything that
is not packaged. This has been the basic premise ever since the check
for unpackaged files was added somewhere around turn of the millenium,
but some loopholes were left around. Yes, %exclude is a loophole.
So 'rm -f' at end of %install for undesired material is the expected fix.
What %check may or may not do has probably never been designed, much
less documented or enforced. It runs after %install yes, so one might
assume it's ok/supposed to use what's in buildroot, but then it runs
from the build directory, not buildroot.
The way I see it, %check should be able to use/access buildroot
contents, but it certainly should not write there. That might be
something to even enforce one day.
When this change was introduced upstream in November 2020, I've analyzed
the impact on Fedora packages. Bare in mind that the data is 4+ months old.
https://github.com/rpm-software-management/rpm/pull/1442#issuecomment-731554917
- 1675 packages had %exclude in the spec file
- 261 packages FTBFS for unrelated reason (incl. a very limited timeout)
- 1414 packages actually tested
- 537 packages built successfully, that is ~38%
- 877 packages failed with unpackaged files, that is ~62%, list attached
When I extrapolate the numbers to compensate the unrelated FTBFS, that's
likely more than 1000 affected Fedora packages.
OTOH ~500 packages are generated rubygem-* packages, automatically fixable.
I'd like to know how are the affected packages supposed to migrate to
RPM 4.17 behavior, especially if they cannot remove the files in
%install prior to %check. Are they supposed to remove the files at the
end of %check instead? What if the package is build without %check?
In the order of preference:
1. 'rm -f <unwanted>' at end of %install
2. change %check not to rely on unpackaged files in buildroot
3. short-term, "%_unpackaged_files_terminate_build 0" in the spec with
explanation (similar to eg unsupported architectures)
2. would be case-by-case of course, but an "universal" solution is to
create a private install root inside %check, eg
---
%check
export CHECKROOT=${PWD}/_check
%make_install DESTDIR=${CHECKROOT}
# ...do what you normally do, just replace BUILDROOT with CHECKROOT
---
This way %check will not be messing with the precious to-be-packaged
contents.
More light-weight options will exist on case-by-case basis, eg typically
a cache can be diverted to some other location with environment
variables or such.
Using `s` in entire rawhide just to
compensate this:
- is dangerous for other implications of the setting
- only postpones the problem to a later time (when we will face the
same issue)
It's hardly "dangerous", because the only content that will "leak"
because of it is buildid links, which is what happens now. It doesn't
affect content inclusion, just whether the message is a warning or error.
And for a more specific problem, around ~100 Python packages were
affected when tested, many of them crucial (e.g. dnf), so this problem
will block the upgrade to Python 3.10 if the change lands in Rawhide
before we upgrade Python (which is the current plan) until we fix all
the affected packages (by at least adding `%global
_unpackaged_files_terminate_build 0` to them, which is a tad big hammer,
but it will be our last-resort option).
Which is why suggested the global "%_unpackaged_files_terminate_build 0"
in rawhide: just to move the impact to a less inconvenient time. No
kittens will be killed by doing so.
- Panu -
List of affected Python packages:
https://pagure.io/releng/issue/10072#comment-724315
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
