On 5/15/23 19:33, Charalampos Stratakis wrote:
On Thu, Mar 30, 2023 at 11:56 PM Miro Hrončok <mhroncok@xxxxxxxxxx
<mailto:mhroncok@xxxxxxxxxx>> wrote:
Hello Python packagers.
RPM 4.19 introduces this feature:
https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html <https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html>
I decided to write this email to gather my thoughts. I believe that
with this,
we can turn manual Python extras subpackages like this:
%package -n python3-...
Summary: %{summary}
%description -n python3-... %_description
%pyproject_extras_subpkg -n python3-xxx extra1 extra2
(See
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras <https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras>
for what that means.)
Into something like this:
%package -n python3-...
Summary: %{summary}
%description -n python3-... %_description
...
%install
%pyproject_install
...
%pyproject_generate_extras_subpkgs -n python3-xxx
The %pyproject_generate_extras_subpkgs macro would parse the installed
.dist-info directory to find out what extras are available and generate
subpackages for all of them.
(Obviously, the macro name is open up for discussion.)
Overall this changes the concept of manually opting-in to extras to
manually opting-out. I have a slight preference over the current status
quo, as the packager needs to be aware of the package's extras and
modify the SPEC accordingly, whereas in the event of implementing an
opt-out approach, things could easily go in the way of enabling
everything till something breaks. At the moment, when working on a
package, I am choosing which extras to enable, leading me to also search
about requirements, purpose of each extra etc.
Being able to enable everything and exclude what I don't need, would
lead to the path of least resistance which is basically to enable
everything and not think about potential issues (as there wouldn't be
any at this point). Now I can imagine "fail to install" bugzilla's over
various extras with the extra caveat that in order to fix a package we
would require to carry the Obsoletes tag in the SPEC for two releases.
Coupled with the fact that an auto-generated list of extras makes
difficult to figure out if an extra has been removed in order to fix it
on the package level makes this approach less than ideal in my head.
I like the gain of automation and I have similar worries that it would
be too easy to let packages slip from our radar.
We could employ automation to report to us when an extra disappears.
Currently, a missing extra is reported via `lostpayload` and
`subpackages` inspections of rpminspect run by Zuul CI which end up in
the VERIFY state.
(example:
https://fedora.softwarefactory-project.io/logs//27/27/7967d00bcc29ea4dcb8520cd4af884546225c52f/check/rpm-rpminspect/f47ac28/result.json)
Maybe rpminspect could extend the `subpackages` inspection to behave
differently:
- report OK if the subpackage is properly Obsoleted
- report BAD if not - fail the entire rpminspect status
This however requires to run rpminspect/Zuul on the package update by
opening a pull request. We enable Zuul CI it for all packages added to
the python-packagers-sig, but many updates land in the branches via
direct push.
Of course not all packages have extras and one can reasonably argue that
packagers who are utilizing the %pyproject_extras_subpkg macro would be
well aware of the implications if they change to the new macro.
While this change would definitely make things easier for packagers
who'd like a cleaner SPEC (and continuing on the path that pyproject
macros have set to align the upstream metadata with the rpm generated
metadata) I believe it can be more error prone than the current approach
and the potential risks outweigh the benefits. I'm happy to be proven
wrong of course.
An API would be required to exclude extras:
- that are not useful for other packages
(for example build/development requirements, commonly named dev,
doc or test)
- that have requirements that are not packaged in Fedora
For example (mimicking the API of %pyproject_check_import):
%pyproject_generate_extras_subpkgs -n python3-xxx -e test -e
'nonfree*'
----------------
However, extras are also currently manually passed to
%pyproject_buildrequires:
%generate_buildrequires
%pyproject_buildrequires -x extra1 -x extra2 -x test
It should already be possible to implement automatic extras
discovery in
%pyproject_buildrequires with older RPM versions and allow it to be
used this way:
%generate_buildrequires
%pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*'
RPM macros can only accept short flags, so
<FLAG_TO_ENABLE_ALL_EXTRAS> can
either be -x '*' (if we start treating -x values as globs, which is
backwards
compatible and probably generally useful), or a single-letter switch
such as -a
(but honestly we are running out of meaningful letters).
(When -X is used, <FLAG_TO_ENABLE_ALL_EXTRAS> can probably be
implied. However,
an explicit form needs to exist for packages that don't need to
exclude any
extras at all.)
Eventually, I'd like to make <FLAG_TO_ENABLE_ALL_EXTRAS> the
default, once RPM
4.19 is omnipresent.
----------------
Combined, this would mean that the packager needs to:
1. specify extras that are not supposed to be used as BRs
2. specify extras that are not supposed to be packaged
In the ideal word (2) is a superset of (1).
Should %pyproject_generate_extras_subpkgs somehow inherit the -Xes from
%pyproject_buildrequires?
When a package has extra1, extra2, nonfree1, nonfree2 and test
extras, one
could do:
%generate_buildrequires
%pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*'
...
%pyproject_install
...
%pyproject_generate_extras_subpkgs -X test
That would mean:
- extra1 is BRed and packaged
- extra2 is BRed and packaged
- test is BRed but not packaged
- nonfree1 is neither
- nonfree2 is neither
Your thoughts on the implementation are quite sound and consistent, I
don't really have any comments on that as you have taken all the
possibilities into account and provided examples. If those thoughts move
forward with an implementation I'll provide more comprehensive feedback
on that.
Also %pyproject_generate_extras_subpkgs somehow inheriting the excluded
extras from %pyproject_buildrequires is reasonable, I dislike the
archaic %global approach as well.
----------------
Alternatively the information could be supplied by %globals:
%global _python_ignored_extras nonfree*
%global _python_unpackaged_extras test
However, I somehow dislike this approach.
----------------
I'd appreciate your thoughts on the matter.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
_______________________________________________
packaging mailing list -- packaging@xxxxxxxxxxxxxxxxxxxxxxx
<mailto:packaging@xxxxxxxxxxxxxxxxxxxxxxx>
To unsubscribe send an email to
packaging-leave@xxxxxxxxxxxxxxxxxxxxxxx
<mailto:packaging-leave@xxxxxxxxxxxxxxxxxxxxxxx>
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
<https://docs.fedoraproject.org/en-US/project/code-of-conduct/>
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
<https://fedoraproject.org/wiki/Mailing_list_guidelines>
List Archives:
https://lists.fedoraproject.org/archives/list/packaging@xxxxxxxxxxxxxxxxxxxxxxx <https://lists.fedoraproject.org/archives/list/packaging@xxxxxxxxxxxxxxxxxxxxxxx>
Do not reply to spam, report it:
https://pagure.io/fedora-infrastructure/new_issue
<https://pagure.io/fedora-infrastructure/new_issue>
--
Regards,
Charalampos Stratakis
Senior Software Engineer
Python Maintenance Team, Red Hat
_______________________________________________
packaging mailing list -- packaging@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to packaging-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/packaging@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
--
Karolina Surma (she/her/hers)
Software Engineer
Python Maintenance Team, Red Hat
_______________________________________________
packaging mailing list -- packaging@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to packaging-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/packaging@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
