On 28. 02. 20 17:58, Adam Williamson wrote:
On Tue, 2020-01-14 at 01:29 +0100, Miro Hrončok wrote:
On 13. 01. 20 22:54, Adam Williamson wrote:
On Sun, 2020-01-12 at 23:24 +0100, Miro Hrončok wrote:
Please don't mention it as required or even necessarily recommended, as
it may not be. I intentionally set up my projects such that tox is
appropriate for upstream CI, not distribution package checking, and
intend distribution packages to run setup.py test.
The %tox macro runs the tests in current environment.
See https://github.com/fedora-python/tox-current-env/ for details.
It's not about the environment. It's about what they actually *do*. In
my projects, tox runs linters, coverage checks and unit tests; setup.py
test only runs unit tests. This is a convenient separation because I
find it appropriate to run linters and coverage checks on commits/PRs
for upstream, but there is no point in running them on package builds.
I agree with everything you said. Taking what we have, ideally we should
convince upstreams to only run tests with default toxenv, not linters.
We also don't have a tox-standard to only run offline testes etc. Maybe one day...
At the same time, `setup.py test` is deprecated.
Sigh. Always nice when people deprecate perfectly well-working
workflows out from underneath you. :/
Right. Two things to consider:
- the deprecation period will most likely last forever
- setup.py files will eventually disappear anyway
So I looked into all of this stuff a bit last night. My conclusions so
far:
* It would be really nice if there were a short Idiot's Guide with a
sample project and RPM spec that puts everything together, as opposed
to me winding up with a browser full of tabs open to
https://src.fedoraproject.org/rpms/pyproject-rpm-macros/tree/master and
https://www.python.org/dev/peps/pep-0517/ and
https://hynek.me/articles/sharing-your-labor-of-love-pypi-quick-and-dirty/
etc. Especially since...
This is all "beta" and provisional, but we'll write a guide once we finish the
%files section macros. In the meantime, see specs in
https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/master/f/tests for
examples.
Yes, we could use some docs about this. No, we don't have them yet. There is also:
[RFE] Documentation about pyproject.toml file
https://bugzilla.redhat.com/show_bug.cgi?id=1739847
* ...the Shiny New Stuff does not appear to be available on EPEL *at
all* yet - not even EPEL 8. This makes it a bit of a non-starter if you
want to use the same spec file bits across Fedora and EPEL. I realize
it may be impractical/impossible to backport everything to EPEL 7, and
am getting close to the point where I throw in the towel and drop
Python 2 support from my projects' master branches and hang the EPEL 7
package repos out on a branch, but EPEL 8 would be nice and ought to be
possible?
This stands in a way of having this in EPEL 8, AFAIK:
- no automatic RPM buildrequires
- ancient pip (PEP 517 support was added in 19.0, we have 9.0.3)
- ancient setuptools (40.8 is needed, we have 39.2.0)
- no tox (but it's being packaged for EPEL8 as we speak)
How are we supposed to do progress in Fedora when people dislike it when it is
not included in EPEL?
* Unless I'm missing something, what you suggested for tox - "ideally
we should convince upstreams to only run tests with default toxenv" -
actually seems weirdly difficult to implement. AFAIK none of the
official or unofficial tox docs I can find really cover the idea of
having an environment that's *defined* but is not *default*. It seems
to be virtually universal practice with tox that you put every
environment in `envlist`...and if you just run `tox` without any `-e`
argument or special env var, it runs every environment in `envlist`.
People seem to assume all environments will be default, and if you want
to run fewer than 'all of them' you filter with `-e` or whatever.
What I meant with "ideally we should convince upstreams to only run tests with
default toxenv, not linters" was this:
`tox -e py37` should only run tests
`tox -e py38` should only run tests
`tox -e py39` should only run tests
Linters should run in `tox -e lint`, or `tox -e py38-lint`.
%tox runs `tox -e py3X` by default.
I *did* figure out a way to do this, but it's something I never really
saw documented anywhere, and I eventually hit a roadblock with using it
in the CI setup I made. You have to make a `tox.ini` like this:
[tox]
envlist = py{27,36,37,38,39}
[testenv]
deps =
-r{toxinidir}/install.requires
-r{toxinidir}/tests.requires
ci: -r{toxinidir}/ci.requires
commands =
py.test
ci: py.test --cov-report term-missing --cov-report xml --cov openqa_client
ci: diff-cover coverage.xml --fail-under=90
ci: diff-quality --violations=pylint --fail-under=90
setenv =
PYTHONPATH = {toxinidir}
by having `ci` directives in `deps` and `commands` like that, you sort
of imply the existence of environments like `py27-ci` without ever
explicitly declaring them, and tox itself is OK with this. You can run
`tox -epy{27,36,37,38,39}-ci` and it'll do what you (maybe) expect -
it'll read ci.requires and run the additional commands. But if you just
run `tox` it'll run the py{27,36,37,38,39} environments without the
additional `ci` bits.
This definitely seems to be a bit 'off the beaten track', though, like
I said - it's not explicitly documented anywhere I could find (most
documentation of the whole generative environments thing assumes all
the environments will be declared up in `envlist`) and commands like
`tox -l` and `tox -a` may not do what you expect.
The showstopper I ultimately hit was in an extension on top of tox,
`tox-gh-actions`, which is a convenience thing for using tox with
GitHub Actions:
https://github.com/ymyzk/tox-gh-actions/issues/11
(the project I'm using as a testbed for this stuff is hosted in github,
and Actions seemed like the easiest way to set up CI). It seems like
this 'implicit non-default environment' thing completely trips up tox-
gh-actions; when I added this section to `tox.ini` to try and make it
use the `ci` environments:
[gh-actions]
python =
2.7: py27
3.6: py36-ci
3.7: py37-ci
3.8: py38-ci
3.9: py39-ci
it would run, but each actual test run seemed not to actually call tox
at all, it just did apparently nothing and then 'passed'.
So in the end I gave up and wound up just making the `-ci` environments
the defaults, explicitly declared in envlist, and figuring I'd have the
spec file use `-e` to explicitly run the non-ci environments.
That's what %tox does.
But then
I didn't even do that because of the whole EPEL thing, so so far I've
wound up not using the pyproject macros at all, I'm doing this instead:
%if 0%{?with_python2}
PYTHONPATH=%{buildroot}%{python2_sitelib} py.test
%endif # with_python3
%if 0%{?with_python3}
PYTHONPATH=%{buildroot}%{python3_sitelib} py.test-3
%endif # with_python3
PYTHON!
EPEL!
Sorry but I cannot help you have nice new things if you decide to have old
things. You need to choose - backwards compatibility or new features.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
