On Wed 26 Feb 2014 01:41:36 PM CET Colin Walters wrote: > On Wed, Feb 26, 2014 at 5:01 AM, Stanislav Ochotnicky > <sochotnicky@xxxxxxxxxx> wrote: >> >> Because unit tests are designed to be run as part of the build >> process. It's not impossible to run them *after* the build, but good >> luck making it work reliably across all packages without manual work. >> > The https://wiki.gnome.org/Initiatives/GnomeGoals/InstalledTests > initative, as implemented by gnome-continuous, takes these "unit tests" > as you call them and runs them as what you call integration tests. "I" didn't name them. I used standard names for different testing levels as defined by software engineering bodies. Quoting from SWEBOK: Software testing is usually performed at different levels along the development and maintenance processes. That is to say, the target of the test can vary: a single module, a group of such modules (related by purpose, use, behavior, or structure), or a whole system. Three big test stages can be conceptually distinguished, namely Unit, Integration, and System. No process model is implied, nor are any of those three stages assumed to have greater importance than the other two. > (Personally, I think distinguishing them is a broken idea. No one runs > just one bit of software, they run a tree - a complete system) Your personal opinion goes against what best practices of software engineering have been for at least past decade. Perhaps you'll not hold it against me if I am going to dismissive about your personal opinion in this case. > For example, after glib changes, I rerun the *gtk* tests. After gtk > changes, I rerun *application* tests. During making glib changes you should run glib unit tests to have some basic level of assurance you didn't introduce regressions or unwanted changes. After that you can run integration/system tests to ensure that gtk or application tests still pass. Simply put unit, integration and system tests are run during different phases of development or maintenance. > This simple change of taking existing valuable tests that were run at > once most per build and turning them into something run 50 or more > times a day made them much more valuable. It also revealed many of > them were full of race conditions... It's great that your change helped with discovering issues but perhaps your original testsuite was mixing different levels of testing in the same code. Unit tests are supposed to be quick, dirty solutions using mock objects and other "hacks" to allow of testing with small granularity. I could probably go on but this would quickly become off topic for fedora-devel and there's tons of literature on introduction to software testing. -- Stanislav Ochotnicky <sochotnicky@xxxxxxxxxx> Software Engineer - Developer Experience PGP: 7B087241 Red Hat Inc. http://cz.redhat.com
Attachment:
pgpKNz5euZ7ja.pgp
Description: PGP signature
-- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
