Chuck R. Anderson wrote:
On Sun, Apr 10, 2005 at 10:57:36PM -0400, Mike A. Harris wrote:
Chuck R. Anderson wrote:
There is a respin done everyday. It is called rawhide, and on good
days you can install it from scratch.
That doesn't test the installer case very well. Most people do not
do anaconda installs of random rawhide builds.
Why not? I do anaconda installs of random rawhide builds all the time
by booting the files in isolinux/ using grub. The only bit that
doesn't get tested is the actual ISO generation. If most people don't
know this can be done, perhaps it should be publicized more.
I think Robert's suggestion is completely rational.
I don't see the need for a full iso set to be generated that often.
Test releases are just "stable" points in the rawhide tree that happen
about once a month. I think this is often enough for full iso sets.
Random rawhide installs provide one type of test coverage. Official
test releases provide a different type of test coverage.
As someone on the receiving end of Fedora Core bug reports, I can
categorically state that the number of bug reports received over
time increases visibly when an official "testN" release occurs,
compared to in-between test releases. This would clearly seem to
indicate that a much much larger number of people test our official
"testN" releases and report bugs than do installer testing via
random rawhide builds.
Maintaining X - the only time I ever see incoming bug reports about
X not working in anaconda, is either immediately following an official
test release, or immediately following an official OS release.
During normal rawhide builds for general development, and in between
the test builds, I *never* see "X hangs during anaconda" bug reports
from people using rawhide installs. My conclusion is that official
test releases provide us with the largest cross section beta testing
compared to the testing done by a much smaller subset of users with
random rawhide builds.
Of course, an alternate conclusion could be that somehow X bugs
never trigger in rawhide installs, and only trigger in official
test releases and official OS releases... but the probability of
that being true is slim.
So blowing an official beta test release due to some nasty bug
that affects a large body of testers, IMHO very much does cause
us to lose a *lot* of real world testing. Plus, since more people
install official test releases, if they get burned badly by one of
our test releases, it theoretically might increase the likelyhood
that they'll be less quick to test the next test release in case
it is bad too.
Finally... If the test coverage we got from people who track rawhide
and/or do rawhide ISO installs was large enough test coverage for
the installer and related environment - we wouldn't likely bother
making "test" releases, because we'd get the test coverage we want
anyway.
For some strange reason, we do however... ;o)