On Wednesday, 22 July 2020 at 14:49, Petr Šplíchal wrote: > Hi! > > I am writing today to share a proposal for how we can make it > easier to enable tests in Fedora CI. > > We've heard the user story mentioned in the subject many times. > Fedora developers have frequently mentioned the difficulties which > they face with enabling tests in Fedora CI. Some people don't > understand the complex yaml syntax. Other people find that they > can't easily reproduce issues revealed by Fedora CI on their > laptops. People have asked us to provide a tool for local testing: > > https://pagure.io/fedora-ci/general/issue/4 > > A few of us have been working on a tool which we hope will improve > the situation considerably. The proof of concept below shows how > everyday developer tasks (related to testing) might look in the > near future: > > Enable basic smoke test for my component: > > dnf install -y tmt > tmt init --template mini > vim plans/example.fmf > git add . && git commit -m tests && git push > > Create a new test based on a shell/beakerlib template: > > tmt test create --template shell tests/basic > tmt test create --template beakerlib tests/advanced > > Run all/selected tests safely in a virtual machine: > > tmt run > tmt run test --name smoke I'd say it's good to have such option, but for quick testing a full VM is often too heavy and slow to boot. Even a container takes time to set up and run. My primary environment would be local. > Execute tests in my preferred environment: > > tmt run --all provision --how container > tmt run --all provision --how virtual > tmt run --all provision --how local > tmt run --all provision --how ... I find it difficult to guess what the above commands actually do. Do they run the tests or do they provision environments to run the tests? Or both? > I would like to ask for your feedback about this. What do you > think? Is this something that would be useful for you? What do > you think of the syntax? We would love to hear back from you > concerning what essential features you might expect from such a > tool. It would also be helpful to know if there is anything that > should be changed early in the tool design. > > For more examples and frequently asked questions have a look here: > > https://tmt.readthedocs.io/en/latest/examples.html > https://docs.fedoraproject.org/en-US/ci/tmt/ > > Looking forward to your feedback and ideas! :) tmt examples are overcomplicated and not explained in enough detail. What is "metadata" and why do you need 4 "levels" of it? What are "tiers" for? Which items are mandatory and which are optional? What's "relevancy"? Why is there "extra-summary" in addition to "summary"? What's "extra-task"? What are "plans" for? What are "stories" for? You're using a lot of terms as if everyone knows what they mean in your context. Regards, Dominik -- Fedora https://getfedora.org | RPM Fusion http://rpmfusion.org There should be a science of discontent. People need hard times and oppression to develop psychic muscles. -- from "Collected Sayings of Muad'Dib" by the Princess Irulan _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
