On Thu, May 21, 2020 at 7:04 PM Richard Shaw <hobbes1069@xxxxxxxxx> wrote: > > On Thu, May 21, 2020 at 11:56 AM Aleksandra Fedorova <alpha@xxxxxxxxxxxx> wrote: >> >> On Thu, May 21, 2020 at 6:30 PM Richard Shaw <hobbes1069@xxxxxxxxx> wrote: >> > >> > On Thu, May 21, 2020 at 10:37 AM Przemo Firszt <przemo@xxxxxxxxx> wrote: >> >> >> >> "FreeCAD -t 0" performs approx 470 tests. No GUI required. Example >> >> output starts here: >> >> https://travis-ci.org/github/FreeCAD/FreeCAD/jobs/689681966#L9103 >> > >> > >> > May need some work to get working from inside mock: >> > >> > # ./FreeCAD -t 0 >> > FreeCAD 0.18, Libs: 0.18RUnknown >> > © Juergen Riegel, Werner Mayer, Yorik van Havre 2001-2019 >> > ##### #### ### #### >> > # # # # # # >> > # ## #### #### # # # # # >> > #### # # # # # # # ##### # # >> > # # #### #### # # # # # >> > # # # # # # # # # ## ## ## >> > # # #### #### ### # # #### ## ## ## >> > >> > Aborted (core dumped) >> >> Please don't run it inside mock. >> >> RPM %check tests are good for unit testing, but they should тще be >> used for extensive integration test suites. >> The build environment is very different from the target user system. >> It has build requirements installed it may have side effects on the >> test you run. >> Also you would be testing the binary in you working directory, where >> you have just executed `make` command. >> >> To run tests in the environment which resembles the actual user >> environment we have the STR framework and gating infrastructure [1]. >> >> To run the command above as the integration test you need to put >> tests/tests.yml file in dist-git repo with the following content: >> >> - hosts: localhost >> roles: >> - role: standard-test-basic >> tags: >> - classic >> tests: >> - simple: >> dir: . >> run: "FreeCAD -t 0" >> >> Note how here you will be using the install FreeCAD binary, not the local one. > > > Is this only run on "real" builds, or is it possible to run locally or for scratch builds? We run it for pull-requests and for "real builds. It runs in the following way: 1) we take latest Fedora Rawhide qcow image, 2) we run virtual machine via qemu-kvm from it, 3) inside the vm we install the package, 4) we run the command defined by "run" key in that YAML file. And it is all orchestrated by Ansible. It is possible to reproduce the test locally with all the CI wrappers around it, but I wouldn't recommend it. To debug the test I would just run the VM with the same Rawhide image [1], install the package and run the test command manually. The exact command we use to run qemu can be seen in default_provisioners.log [2] Now, as soon as you figure out the test itself and you know that it is working, you can verify that it works when wrapped in CI orchestration. You can do that by submitting pull-request with this test configuration via src.fedoraproject.org. On pull request we do additional step: we take the patch from the pull request and submit the scratch build to Koji. Once scratch build is done, we run the test as described above. The test result is then published on the pull request page under the name Fedora CI. See the right side labels on the example page [3] [1] https://jenkins-continuous-infra.apps.ci.centos.org/job/fedora-rawhide-image-test/lastSuccessfulBuild/artifact/Fedora-Rawhide.qcow2 [2] https://jenkins-continuous-infra.apps.ci.centos.org/job/fedora-rawhide-pr-pipeline/3623/artifact/nvr-verify/logs/default_provisioners.log [3] https://src.fedoraproject.org/rpms/libdnf-plugin-swidtags/pull-request/5 -- Aleksandra Fedorova bookwar _______________________________________________ devel mailing list -- devel@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to devel-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@xxxxxxxxxxxxxxxxxxxxxxx
