A number of pieces of software included in Fedora have automated regression tests included with their source. I think it would be handy if there was a way to easily install and run these tests in a uniform fashion. One way to accomplish this would be to have the spec optionally build a packagename-regression rpm, which includes the scripts and data needed to perform the automated regression testing. During testing users could choose to install all the available regression rpms, and run the tests. Even a fairly limited set of tests would likely do a good job at finding problems due to lower level infrastructure changes (gcc bugs, glibc, kernel, etc). Furthermore, distribution wide uniform regression testing would encourage the creation of regression test suites for all packages... Right now, no one will likely run a regression suite for 'bc', so no one bothers writing one... but it would get run as part of a distribution wide testing. If all the silly little apps had some basic testing we would be much more likely to catch bugs in the system as a whole. It would also be useful to track the performance of these tests, so that performance regressions on various platforms could be spotted and addressed early on, and not just reactively. Am I totally off the mark here? Has work been done on this front already?
