On 12/02/2009 03:16 PM, James Laska wrote:
> If I'm summarizing the pain correctly ... the push back seems to come
> from providing testing before development is ready to accept bug
> reports?
>
> Does this only happen during early development?
No. In fact, a part of this *is* due the difficulty in testing loader
changes. Pretty often the (current) workflow goes like:
1) joe writes a loader change that's complex
2) joe compile tests it, tries to unit test code snippets isolated into
their own tiny (transient) test harnesses where possible (which it
often is not).
3) joe fixes whatever problems are found
4) joe submits the change for review on a-d-l
5) a-d-l review finds some problems, misses others
6) if a-d-l review finds problems, joe goes back to step 2
7) else, joe commits the change
8) 0 or more days later a build gets done by joe or somebody else
9) rawhide composes with the new build
10) rawhide is broken because of one character in a string someplace
being wrong!
11) bob comes in in the morning and does a test install with rawhide
12) bob fixes the error, goes through steps 1-7 again (or maybe
just commits a trivial change, but generally not these days)
13) pile-o-bug-reports arrives, almost all of them after the bug is fixed
14) mike spends 8 hours triaging bug reports that are completely
irrelevant
Note how there's no set schedule on step 8.
This happens at all times during development. We can alleviate some of
the problem by writing/improving tools, for example to build/update
anaconda's initrd.img, but I don't honestly believe that will cut down
more than, say, half of this problem. It also doesn't really help
with of the /combined/ tree (that is, with changes other developers are
working on), with arches that aren't what we're using on the desktop,
or with e.g. CD media. For those, there's not much you can do without
just building it in koji and punging up a tree. And we don't really
feel it's worth pushing to testers, no matter what quality of tester
or test methodology, until we've done some simple trials first.
The new way, including some feedback from this thread, is more like this:
1) steps 1-8 from above happen
9) somebody on the anaconda team composes a tree[1]
10) we (the anaconda team) smoke test it and go through steps 11 and 12
above
13) one of us asks for a tree to be composed for testing
14) we (I guess rel-eng does this bit) put some images somewhere for
testing [details needed here]
15) if they're not catastrophic we replace the LNG images with those
16) we (the anaconda team) go back to step 1.
[1]: I'm most likely going to automate this so we automatically get
x86_64 and i386 trees composed on cutlet (our local mirror) any
time it sees a new anaconda package in rawhide.
> Meaning, after Alpha/Beta is having nightly built rawhide install
> images still a problem? When in the schedule [1] would be an
> acceptable time to start providing test feedback?
I think maybe after beta could work if we feel it's really necessary;
it's not wholly unreasonable to expect us (the anaconda team) to be a
bit more rigorous in composing our own trees and testing before (and
after) step 7. But you've got to realize that that's basically just
reverting to the original process above, which still has step 8 with no
set schedule. So it's not like that's really different - you're just
seeing the same installer over and over.
> Another point, I think we can't limit the discussion to just
> anaconda-devel. There are a *lot* of other critical components pulled
> into the install images that are developed outside that group.
We didn't ever limit this discussion to anaconda-devel. Actually,
I don't think we've discussed this there at all.
--
Peter
Old MacDonald had an agricultural real-estate tax abatement.
--
fedora-test-list mailing list
fedora-test-list@xxxxxxxxxx
To unsubscribe:
https://www.redhat.com/mailman/listinfo/fedora-test-list
