Re: Tox automation in packaging macros

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 28. 02. 20 17:58, Adam Williamson wrote:
On Tue, 2020-01-14 at 01:29 +0100, Miro Hrončok wrote:
On 13. 01. 20 22:54, Adam Williamson wrote:
On Sun, 2020-01-12 at 23:24 +0100, Miro Hrončok wrote:
Please don't mention it as required or even necessarily recommended, as
it may not be. I intentionally set up my projects such that tox is
appropriate for upstream CI, not distribution package checking, and
intend distribution packages to run setup.py test.

The %tox macro runs the tests in current environment.

See https://github.com/fedora-python/tox-current-env/ for details.

It's not about the environment. It's about what they actually *do*. In
my projects, tox runs linters, coverage checks and unit tests; setup.py
test only runs unit tests. This is a convenient separation because I
find it appropriate to run linters and coverage checks on commits/PRs
for upstream, but there is no point in running them on package builds.

I agree with everything you said. Taking what we have, ideally we should
convince upstreams to only run tests with default toxenv, not linters.

We also don't have a tox-standard to only run offline testes etc. Maybe one day...

At the same time, `setup.py test` is deprecated.

Sigh. Always nice when people deprecate perfectly well-working
workflows out from underneath you. :/

Right. Two things to consider:

   - the deprecation period will most likely last forever
   - setup.py files will eventually disappear anyway

So I looked into all of this stuff a bit last night. My conclusions so
far:

* It would be really nice if there were a short Idiot's Guide with a
sample project and RPM spec that puts everything together, as opposed
to me winding up with a browser full of tabs open to
https://src.fedoraproject.org/rpms/pyproject-rpm-macros/tree/master and
https://www.python.org/dev/peps/pep-0517/ and
https://hynek.me/articles/sharing-your-labor-of-love-pypi-quick-and-dirty/
etc. Especially since...

This is all "beta" and provisional, but we'll write a guide once we finish the %files section macros. In the meantime, see specs in https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/master/f/tests for examples.

Yes, we could use some docs about this. No, we don't have them yet. There is also:

[RFE] Documentation about pyproject.toml file
https://bugzilla.redhat.com/show_bug.cgi?id=1739847

* ...the Shiny New Stuff does not appear to be available on EPEL *at
all* yet - not even EPEL 8. This makes it a bit of a non-starter if you
want to use the same spec file bits across Fedora and EPEL. I realize
it may be impractical/impossible to backport everything to EPEL 7, and
am getting close to the point where I throw in the towel and drop
Python 2 support from my projects' master branches and hang the EPEL 7
package repos out on a branch, but EPEL 8 would be nice and ought to be
possible?

This stands in a way of having this in EPEL 8, AFAIK:

 - no automatic RPM buildrequires
 - ancient pip (PEP 517 support was added in 19.0, we have 9.0.3)
 - ancient setuptools (40.8 is needed, we have 39.2.0)
 - no tox (but it's being packaged for EPEL8 as we speak)

How are we supposed to do progress in Fedora when people dislike it when it is not included in EPEL?

* Unless I'm missing something, what you suggested for tox - "ideally
we should convince upstreams to only run tests with default toxenv" -
actually seems weirdly difficult to implement. AFAIK none of the
official or unofficial tox docs I can find really cover the idea of
having an environment that's *defined* but is not *default*. It seems
to be virtually universal practice with tox that you put every
environment in `envlist`...and if you just run `tox` without any `-e`
argument or special env var, it runs every environment in `envlist`.
People seem to assume all environments will be default, and if you want
to run fewer than 'all of them' you filter with `-e` or whatever.

What I meant with "ideally we should convince upstreams to only run tests with default toxenv, not linters" was this:

 `tox -e py37` should only run tests
 `tox -e py38` should only run tests
 `tox -e py39` should only run tests

Linters should run in `tox -e lint`, or `tox -e py38-lint`.

%tox runs `tox -e py3X` by default.

I *did* figure out a way to do this, but it's something I never really
saw documented anywhere, and I eventually hit a roadblock with using it
in the CI setup I made. You have to make a `tox.ini` like this:

[tox]
envlist = py{27,36,37,38,39}

[testenv]
deps =
     -r{toxinidir}/install.requires
     -r{toxinidir}/tests.requires
     ci: -r{toxinidir}/ci.requires

commands =
     py.test
     ci: py.test --cov-report term-missing --cov-report xml --cov openqa_client
     ci: diff-cover coverage.xml --fail-under=90
     ci: diff-quality --violations=pylint --fail-under=90

setenv =
     PYTHONPATH = {toxinidir}

by having `ci` directives in `deps` and `commands` like that, you sort
of imply the existence of environments like `py27-ci` without ever
explicitly declaring them, and tox itself is OK with this. You can run
`tox -epy{27,36,37,38,39}-ci` and it'll do what you (maybe) expect -
it'll read ci.requires and run the additional commands. But if you just
run `tox` it'll run the py{27,36,37,38,39} environments without the
additional `ci` bits.

This definitely seems to be a bit 'off the beaten track', though, like
I said - it's not explicitly documented anywhere I could find (most
documentation of the whole generative environments thing assumes all
the environments will be declared up in `envlist`) and commands like
`tox -l` and `tox -a` may not do what you expect.

The showstopper I ultimately hit was in an extension on top of tox,
`tox-gh-actions`, which is a convenience thing for using tox with
GitHub Actions:

https://github.com/ymyzk/tox-gh-actions/issues/11

(the project I'm using as a testbed for this stuff is hosted in github,
and Actions seemed like the easiest way to set up CI). It seems like
this 'implicit non-default environment' thing completely trips up tox-
gh-actions; when I added this section to `tox.ini` to try and make it
use the `ci` environments:

[gh-actions]
python =
     2.7: py27
     3.6: py36-ci
     3.7: py37-ci
     3.8: py38-ci
     3.9: py39-ci

it would run, but each actual test run seemed not to actually call tox
at all, it just did apparently nothing and then 'passed'.

So in the end I gave up and wound up just making the `-ci` environments
the defaults, explicitly declared in envlist, and figuring I'd have the
spec file use `-e` to explicitly run the non-ci environments.

That's what %tox does.

But then
I didn't even do that because of the whole EPEL thing, so so far I've
wound up not using the pyproject macros at all, I'm doing this instead:

%if 0%{?with_python2}
PYTHONPATH=%{buildroot}%{python2_sitelib} py.test
%endif # with_python3
%if 0%{?with_python3}
PYTHONPATH=%{buildroot}%{python3_sitelib} py.test-3
%endif # with_python3

PYTHON!

EPEL!

Sorry but I cannot help you have nice new things if you decide to have old things. You need to choose - backwards compatibility or new features.

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
_______________________________________________
devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Fedora Announce]     [Fedora Users]     [Fedora Kernel]     [Fedora Testing]     [Fedora Formulas]     [Fedora PHP Devel]     [Kernel Development]     [Fedora Legacy]     [Fedora Maintainers]     [Fedora Desktop]     [PAM]     [Red Hat Development]     [Gimp]     [Yosemite News]

  Powered by Linux