Some thoughts: >> Where to keep tests? a/ in current dist-git for related components >> (problem with sharing parts of code, problem where to keep tests >> related for more components) b/ in separate git with similar >> functionality as dist-git (needs new infrastructure, components are >> not directly connected with tests, won't make mess in current >> dist-git) c/ in current dist-git but as ordinary components (no new >> infrastructure needed but components are not directly connected >> with tests) >> >> How to deliver tests? a/ just use them directly from git (we need >> to keep some metadata for dependencies anyway) b/ package them as >> RPMs (we can keep metadata there; e.g. Taskotron will run only >> tests that have "Provides: ci-tests(mariadb)" after mariadb is >> built; we also might automate packaging tests to RPMs) To answer both of these, the plan is to keep taskotron tasks in their own git repo; currently this is at (0). To run the tasks, taskotron sets up a disposable task client and then git clones the task to be run. This solves the issue of delivery and allows a continuous integration-like solution. >> Structure for tests? a/ similar to what components use (branches >> for Fedora versions) b/ only one branch Test maintainers should be >> allowed to behave the same as package maintainers do -- one likes >> keeping branches the same and uses "%if %fedora" macros, someone >> else likes specs clean and rather maintain more different branches) >> -- we won't find one structure that would fit all, so allowing both >> ways seems better. This is something we'll need to figure out, but, I suspect git branches will be involved. >> Which framework to use? People have no time to learn new things, so >> we should let them to write the tests in any language and just >> define some conventions how to run them. You'll need to at least define the task in a yaml file, and output will need to be TAP. The example task is at (1). > TAP (Test Anything Protocol) FTW. It really makes sense when you're > trying to combine tests from multiple different languages, testing > frameworks, etc. > > Stef > Indeed, which is why we settled on it. John. (0) https://bitbucket.org/fedoraqa (1) https://bitbucket.org/fedoraqa/task-rpmlint -- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
