On Tue, 2020-01-14 at 01:29 +0100, Miro Hrončok wrote: > On 13. 01. 20 22:54, Adam Williamson wrote: > > On Sun, 2020-01-12 at 23:24 +0100, Miro Hrončok wrote: > > > > Please don't mention it as required or even necessarily recommended, as > > > > it may not be. I intentionally set up my projects such that tox is > > > > appropriate for upstream CI, not distribution package checking, and > > > > intend distribution packages to run setup.py test. > > > > > > The %tox macro runs the tests in current environment. > > > > > > See https://github.com/fedora-python/tox-current-env/ for details. > > > > It's not about the environment. It's about what they actually *do*. In > > my projects, tox runs linters, coverage checks and unit tests; setup.py > > test only runs unit tests. This is a convenient separation because I > > find it appropriate to run linters and coverage checks on commits/PRs > > for upstream, but there is no point in running them on package builds. > > I agree with everything you said. Taking what we have, ideally we should > convince upstreams to only run tests with default toxenv, not linters. > > We also don't have a tox-standard to only run offline testes etc. Maybe one day... > > > > At the same time, `setup.py test` is deprecated. > > > > Sigh. Always nice when people deprecate perfectly well-working > > workflows out from underneath you. :/ > > Right. Two things to consider: > > - the deprecation period will most likely last forever > - setup.py files will eventually disappear anyway So I looked into all of this stuff a bit last night. My conclusions so far: * It would be really nice if there were a short Idiot's Guide with a sample project and RPM spec that puts everything together, as opposed to me winding up with a browser full of tabs open to https://src.fedoraproject.org/rpms/pyproject-rpm-macros/tree/master and https://www.python.org/dev/peps/pep-0517/ and https://hynek.me/articles/sharing-your-labor-of-love-pypi-quick-and-dirty/ etc. Especially since... * ...the Shiny New Stuff does not appear to be available on EPEL *at all* yet - not even EPEL 8. This makes it a bit of a non-starter if you want to use the same spec file bits across Fedora and EPEL. I realize it may be impractical/impossible to backport everything to EPEL 7, and am getting close to the point where I throw in the towel and drop Python 2 support from my projects' master branches and hang the EPEL 7 package repos out on a branch, but EPEL 8 would be nice and ought to be possible? * Unless I'm missing something, what you suggested for tox - "ideally we should convince upstreams to only run tests with default toxenv" - actually seems weirdly difficult to implement. AFAIK none of the official or unofficial tox docs I can find really cover the idea of having an environment that's *defined* but is not *default*. It seems to be virtually universal practice with tox that you put every environment in `envlist`...and if you just run `tox` without any `-e` argument or special env var, it runs every environment in `envlist`. People seem to assume all environments will be default, and if you want to run fewer than 'all of them' you filter with `-e` or whatever. I *did* figure out a way to do this, but it's something I never really saw documented anywhere, and I eventually hit a roadblock with using it in the CI setup I made. You have to make a `tox.ini` like this: [tox] envlist = py{27,36,37,38,39} [testenv] deps = -r{toxinidir}/install.requires -r{toxinidir}/tests.requires ci: -r{toxinidir}/ci.requires commands = py.test ci: py.test --cov-report term-missing --cov-report xml --cov openqa_client ci: diff-cover coverage.xml --fail-under=90 ci: diff-quality --violations=pylint --fail-under=90 setenv = PYTHONPATH = {toxinidir} by having `ci` directives in `deps` and `commands` like that, you sort of imply the existence of environments like `py27-ci` without ever explicitly declaring them, and tox itself is OK with this. You can run `tox -epy{27,36,37,38,39}-ci` and it'll do what you (maybe) expect - it'll read ci.requires and run the additional commands. But if you just run `tox` it'll run the py{27,36,37,38,39} environments without the additional `ci` bits. This definitely seems to be a bit 'off the beaten track', though, like I said - it's not explicitly documented anywhere I could find (most documentation of the whole generative environments thing assumes all the environments will be declared up in `envlist`) and commands like `tox -l` and `tox -a` may not do what you expect. The showstopper I ultimately hit was in an extension on top of tox, `tox-gh-actions`, which is a convenience thing for using tox with GitHub Actions: https://github.com/ymyzk/tox-gh-actions/issues/11 (the project I'm using as a testbed for this stuff is hosted in github, and Actions seemed like the easiest way to set up CI). It seems like this 'implicit non-default environment' thing completely trips up tox- gh-actions; when I added this section to `tox.ini` to try and make it use the `ci` environments: [gh-actions] python = 2.7: py27 3.6: py36-ci 3.7: py37-ci 3.8: py38-ci 3.9: py39-ci it would run, but each actual test run seemed not to actually call tox at all, it just did apparently nothing and then 'passed'. So in the end I gave up and wound up just making the `-ci` environments the defaults, explicitly declared in envlist, and figuring I'd have the spec file use `-e` to explicitly run the non-ci environments. But then I didn't even do that because of the whole EPEL thing, so so far I've wound up not using the pyproject macros at all, I'm doing this instead: %if 0%{?with_python2} PYTHONPATH=%{buildroot}%{python2_sitelib} py.test %endif # with_python3 %if 0%{?with_python3} PYTHONPATH=%{buildroot}%{python3_sitelib} py.test-3 %endif # with_python3 PYTHON! -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
