F38 proposal: Deprecate python-toml (Self-Contained Change proposal)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



https://fedoraproject.org/wiki/Changes/DeprecatePythonToml

This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.


== Summary ==
The {{package|python-toml}} (`python3-toml`) package will be
[https://docs.fedoraproject.org/en-US/packaging-guidelines/deprecating-packages/
deprecated] in [[Releases/38|Fedora 38]]. The
[https://pypi.org/project/toml/ upstream toml package] is considered
unmaintained (see [[#Detailed_Description|description]]) and Python
3.11 contains [https://peps.python.org/pep-0680/ a TOML-reading
library in the standard library]. Existing Fedora packages depend on
{{package|python-toml}}, so we cannot remove it yet. Packagers are
encouraged to work with upstreams to switch to
[https://peps.python.org/pep-0680/
tomllib]/[https://pypi.org/project/tomli/ tomli] for reading toml or
[https://pypi.org/project/tomli/ tomli-w] for writing it. But
{{package|python-toml}} remains available until it is a leaf package,
it will be removed then (possibly not yet in Fedora 38).

== Owner ==
* Name: [[User:Churchyard|Miro Hrončok]]
* Email: mhroncok@xxxxxxxxxx


== Detailed Description ==
The {{package|python-toml}} package is [https://pypi.org/project/toml/
unmaintained upstream]. It does not support the latest TOML standard
and no longer releases newer versions.

We'd like to drop it from Fedora, but several packages still require
it. Before we attempt to remove the package, we need to stop new
packages to (Build)Require `python3-toml`, hence we want to have it
[https://docs.fedoraproject.org/en-US/packaging-guidelines/deprecating-packages/
deprecated]. No new packages that require it may be added to Fedora
and existing packages may not gain new dependencies on it. Requiring
it only `if python3 < 3.11` or similar is allowed because python3 on
Fedora 38 is 3.11.

Packagers are encouraged to switch to an alternative TOML library with
upstream involvement. Downstream-only patches to switch are not
encouraged. Change owners recommend the following alternatives:

* Use the [https://docs.python.org/3.11/library/tomllib.html tomllib]
module from the standard library to read TOML with Python 3.11+.
* Use the {{package|python-tomli}} package to read TOML with an older
version of Python. The `tomllib` module has started as `tomli` and
they share the same API except for the module name.
* Use the {{package|python-tomli-w}} package to write TOML.

Note that <code>repoquery</code> gives many packages depending on
`python3-toml`:

 $ repoquery --repo=rawhide{,-source} --whatrequires python3-toml | wc -l
 443

This is because many packages BuildRequire `(python3dist(toml) if
python3-devel < 3.11)` due to {{package|pyproject-rpm-macros}}.

 $ repoquery --repo=rawhide{,-source} --whatrequires
'(python3dist(toml) if python3-devel < 3.11)' | wc -l
 413

The change owners don't know how to
[https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx/thread/3YA5AVHIM65FRSTLLISY5Y7KNGOS4KYA/
easily filter them out], but when filtered the hard way, this remains:

(Results from 2022-10-05, may contain false positives.)

 $ for pkg in $(repoquery --repo=rawhide{,-source} --whatrequires
python3-toml); do repoquery -q --repo=rawhide{,-source} --requires
$pkg | grep -Fv '(python3dist(toml) if python3-devel < 3.11)' | grep
-Eq '(\(|-)toml\b' && echo $pkg; done
 academic-admin-0:0.5.1-10.fc37.noarch
 academic-admin-0:0.5.1-10.fc37.src
 bandit-0:1.7.4-3.fc37.src
 bst-external-0:0.29.0-1.fc38.src
 cvc4-0:1.8-12.fc37.src
 fedora-license-data-0:1.5-1.fc38.src
 fedora-messaging-0:3.1.0-5.fc38.src
 gi-docgen-0:2022.1-7.fc38.noarch
 gi-docgen-0:2022.1-7.fc38.src
 jrnl-0:3.0-3.fc37.src
 micropipenv-0:1.4.2-1.fc37.noarch
 pre-commit-0:2.20.0-2.fc37.noarch
 pre-commit-0:2.20.0-2.fc37.src
 pylint-0:2.14.4-3.fc37.src
 python-anyconfig-0:0.13.0-3.fc37.src
 python-anymarkup-0:0.8.1-10.fc37.src
 python-anymarkup-core-0:0.8.1-9.fc37.src
 python-ast-monitor-0:0.2.1-1.fc38.src
 python-asttokens-0:2.0.8-1.fc38.src
 python-autopep8-0:1.6.0-5.fc37.src
 python-botocore-0:1.27.86-1.fc38.src
 python-box-0:6.0.2-1.fc38.src
 python-build-0:0.8.0-4.fc37.src
 python-check-manifest-0:0.48-3.fc37.src
 python-deepdiff-0:5.8.2-2.fc37.src
 python-devicely-0:1.1.1-3.fc37.src
 python-elpy-0:1.34.0-8.fc37.src
 python-exoscale-0:0.7.1-4.fc37.src
 python-fasjson-client-0:1.0.7-5.fc38.src
 python-fireflyalgorithm-0:0.3.2-2.fc37.src
 python-interrogate-0:1.5.0-4.fc37.src
 python-jsonpickle-0:2.2.0-4.fc37.src
 python-lsp-black-0:1.2.0-3.fc37.src
 python-matrix-nio-0:0.19.0-6.fc38.src
 python-molecule-podman-0:1.0.1-4.fc37.src
 python-neurom-0:3.1.0-5.fc37.src
 python-niaaml-0:1.1.11-1.fc38.src
 python-niaarm-0:0.2.1-2.fc38.src
 python-niaclass-0:0.1.2-8.fc37.src
 python-nikola-0:8.2.2-4.fc37.src
 python-pendulum-0:2.1.2-8.fc37.src
 python-podman-3:4.2.0-7.fc38.src
 python-pyedflib-0:0.1.30-2.fc37.src
 python-pyqt-feedback-flow-0:0.1.6-3.fc37.src
 python-rich-0:12.6.0-1.fc38.src
 python-rst-linker-0:2.3.1-1.fc38.src
 python-sklearn-nature-inspired-algorithms-0:0.9.0-3.fc37.src
 python-sport-activities-features-0:0.3.6-1.fc38.src
 python-stochastic-0:0.7.0-2.fc37.src
 python-toml-adapt-0:0.2.7-3.fc37.src
 python-usort-0:0.6.3-7.fc37.src
 python-vulture-0:2.6-1.fc38.src
 python3-anymarkup-0:0.8.1-10.fc37.noarch
 python3-autopep8-0:1.6.0-5.fc37.noarch
 python3-box-0:6.0.2-1.fc38.noarch
 python3-exoscale-0:0.7.1-4.fc37.noarch
 python3-fasjson-client-0:1.0.7-5.fc38.noarch
 python3-fedora-messaging-0:3.1.0-5.fc38.noarch
 python3-interrogate-0:1.5.0-4.fc37.noarch
 python3-jaraco-functools-0:3.5.2-1.fc38.noarch
 python3-jinja2-cli-0:0.8.2-3.fc37.noarch
 python3-lsp-black-0:1.2.0-3.fc37.noarch
 python3-nikola-0:8.2.2-4.fc37.noarch
 python3-podman-3:4.2.0-7.fc38.noarch
 python3-sklearn-nature-inspired-algorithms-0:0.9.0-3.fc37.noarch
 python3-toml-adapt-0:0.2.7-3.fc37.noarch
 python3-usort-0:0.6.3-7.fc37.noarch
 python3-vulture-0:2.6-1.fc38.noarch
 rpmlint-0:2.2.0-7.fc38.noarch
 rpmlint-0:2.2.0-7.fc38.src
 sip6-0:6.6.2-2.fc37.src
 sip6-0:6.6.2-2.fc37.x86_64
 teampulls-0:0.2.2-10.fc37.noarch
 trac-tracnav-plugin-0:4.3-7.fc37.src

=== List of components still (Build)Requiring python3-toml ===

* {{package|academic-admin}}
* {{package|bandit}}
* {{package|bst-external}}
* {{package|cvc4}}
* {{package|fedora-license-data}} (false positive, dependency
conditional on python3 < 3.11)
* {{package|fedora-messaging}}
* {{package|gi-docgen}}
* {{package|jrnl}}
* {{package|micropipenv}} (already fixed in the meantime)
* {{package|pre-commit}}
* {{package|pylint}}
* {{package|python-anyconfig}}
* {{package|python-anymarkup}}
* {{package|python-anymarkup-core}}
* {{package|python-ast-monitor}}
* {{package|python-asttokens}}
* {{package|python-autopep8}}
* {{package|python-botocore}}
* {{package|python-box}}
* {{package|python-build}}
* {{package|python-deepdiff}}
* {{package|python-devicely}}
* {{package|python-elpy}}
* {{package|python-exoscale}}
* {{package|python-fasjson-client}}
* {{package|python-fireflyalgorithm}}
* {{package|python-check-manifest}}
* {{package|python-interrogate}}
* {{package|python-jaraco-functools}} (superfluous,
[https://src.fedoraproject.org/rpms/python-jaraco-functools/pull-request/5
proposed fix])
* {{package|python-jinja2-cli}}
* {{package|python-jsonpickle}}
* {{package|python-lsp-black}}
* {{package|python-matrix-nio}}
* {{package|python-molecule-podman}}
* {{package|python-neurom}}
* {{package|python-niaaml}}
* {{package|python-niaarm}}
* {{package|python-niaclass}}
* {{package|python-nikola}}
* {{package|python-pendulum}}
* {{package|python-podman}}
* {{package|python-pyedflib}}
* {{package|python-pyqt-feedback-flow}}
* {{package|python-rich}}
* {{package|python-rst-linker}}
* {{package|python-sklearn-nature-inspired-algorithms}}
* {{package|python-sport-activities-features}}
* {{package|python-stochastic}}
* {{package|python-toml-adapt}}
* {{package|python-usort}}
* {{package|python-vulture}}
* {{package|rpmlint}} (already fixed in the meantime)
* {{package|sip6}}
* {{package|teampulls}}
* {{package|trac-tracnav-plugin}}

Once all dependencies are removed, we plan to retire
{{package|python-toml}}, whether it will be in Fedora 38 (unlikely) or
later.

=== Migrating to tomllib ===

* Remove any `toml` requirements from upstream metadata.
* Change all `toml` imports to `tomllib` imports.
* Change all references of `TomlDecodeError` to `TOMLDecodeError`.
* Open files in binary mode, if passing file objects to `tomllib.load()`.

=== Migrating to tomli ===

* Change any `toml` requirements in upstream metadata to `tomli`.
* Change all `toml` imports to `tomli` imports.
* Change all references of `TomlDecodeError` to `TOMLDecodeError`.
* Open files in binary mode, if passing file objects to `tomli.load()`.

=== Migrating to tomllib on Python 3.11+ and falling back to tomli ===

* Change any `toml` requirements in upstream metadata to
`tomli;python_version<"3.11"`.
* Change all `toml` imports to `sys.version_info`-conditional or `try:
except ImportError:` `tomli`/`tomllib` imports.
* Change all references of `TomlDecodeError` to `TOMLDecodeError`.
* Open files in binary mode, if passing file objects to `tomllib.load()`.

Example of importing:

 try:
     import tomllib
 except ImportError:
     import tomli as tomllib

Or:

 if sys.version_info < (3, 11):
     import tomli as tomllib
 else:
     import tomllib

Upstream example: https://github.com/pytest-dev/pytest/pull/9741

A more complex example that also falls back to `toml` on Python 2.7 or
3.6: https://github.com/tox-dev/tox/pull/2463

A more complex example that supports `pytoml`, `toml`, `tomli` and
`tomllib`: https://github.com/thoth-station/micropipenv/pull/241

=== Migrating to tomllib on Python 3.11+ and falling back to toml ===

Despite `toml` not being maintained upstream, it exists in EPEL 8,
unlike `tomli` (currently).
Some Fedora-upstream projects might need to migrate to `tomllib` but
keep using `toml` on older Pythons.

* Change any `toml` requirements in upstream metadata to
`toml;python_version<"3.11"`.
* Change all `toml` imports to `sys.version_info`-conditional or `try:
except ImportError:` `tomli`/`tomllib` imports.
* Change all references of `TomlDecodeError` to `TOMLDecodeError`
depending on what was imported.
* Open files in binary/text mode depending on what was imported.

 try:
     import tomllib as toml_module

     toml_mode = "rb"
     toml_exception = toml_module.TOMLDecodeError
 except ImportError:
     import toml as toml_module

     toml_mode = "r"
     toml_exception = toml_module.TomlDecodeError

 with open(..., toml_mode) as f:
     try:
         _ = toml_module.load(f)
     except toml_exception as e:
         ...

Example upstream PR: https://github.com/fedora-infra/fedora-messaging/pull/274

=== Migrating to tomli-w ===

* Change any `toml` requirements in upstream metadata to `tomli-w`.
* Change all `toml` imports to `tomli_w` imports.
* Open files in binary mode, if passing file objects to `tomli_w.dump()`.

A more complex example that migrates to `tomli`, `tomllib` and
`tomli-w`: https://github.com/rpm-software-management/rpmlint/pull/905

== Feedback ==

== Benefit to Fedora ==
An upstream dead package will not be depended upon by new packages.

== Scope ==
* Proposal owners: Deprecate `python3-toml`. Work with packagers and
upstream developers to remove the dependency. Monitor the remaining
dependent packages and eventually retire {{package|python-toml}}
(unlikely in Fedora 38).

* Other developers: No action needed. Don't add new dependencies on
`python3-toml`.

* Release engineering: N/A (not a System Wide Change)
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives:

== Upgrade/compatibility impact ==
The package will remain available. Only new packages cannot depend on it.
Once retired, we don't plan to provide <code>python3-toml</code> from
`python3-libs` or `python3-tomli`, because it cannot work as a drop-in
replacement (the Python module has a different name and slightly
different API). The package will eventually be obsoleted by
{{package|fedora-obsolete-packages}} once retired, but that is
unlikely to happen soon.

== How To Test ==
 $ repoquery --repo=rawhide --provides python3-toml
 ...
 deprecated()
 ...

== User Experience ==
No changes.

== Dependencies ==
N/A (not a System Wide Change)

== Contingency Plan ==
* Contingency mechanism: revert the deprecation
* Contingency deadline: Final Freeze
* Blocks release? No

== Documentation ==
N/A (not a System Wide Change)

== Release Notes ==


-- 
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux