On Wed, Feb 26, 2014 at 11:50 AM, Miloslav Trmač <mitr@xxxxxxxx> wrote:
Are you saying that the boot path should have tests, and the less-frequently used parts of the system should be verified by seeing whether any human users notice breakage?
No. First, it's more that in order to run any other tests, the system must boot.
Now of course there's non-test "reporting" stuff like static code analysis, code style checkers...those things make sense in a mock chroot to me. They're not significantly more demanding than a simple build.
I don't think "if it boots, ship it" is an acceptable quality target.
The neat thing is that I can easily define multiple, progressively enhanced quality targets, and I do in fact.
For each ref, I have "buildmaster" in the string. All that happened with this branch is that we stuck the RPMs together. We don't know if it boots.
But quickly after that, we do boot. That tree gets tagged as "smoketested" if it succeeds, and you can track just that tree. It introduces a few minutes of latency.
After smoketested, there's a vast array of potential tests to run. I think it makes sense to have a "baseline" per tree. For example, for server/docker-io, it might mean that docker can start and pull images and run some basic things.
(One could argue for a per-tree smoketested that is this type of thing, but I think it makes sense to have a distinct phase which is the basic fundamental requirements for anything at all, such as having a functional systemd)
-- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
