#175: Improve transfer of previous test results in the installation matrix
-------------------------+--------------------------------------------------
Reporter: kparal | Owner: rhe
Type: enhancement | Status: new
Priority: major | Milestone: Fedora 15
Component: Wiki | Version:
Keywords: |
-------------------------+--------------------------------------------------
Currently in our installation matrix there are some test results which are
from "anonymous" source (no name filled in). Example:
https://fedoraproject.org/wiki/Test_Results:Fedora_15_Alpha_RC2_Install
I was little confused by that, but rhe explained me that those are the
test results that were transfered from the previous test result (i.e. from
F15 Alpha RC1 in this case). We already agreed if would be better if we
added <ref> with explanation to each such result, so other testers
understand the meaning of it (I suppose many of them might be confused
same as I was). Still, I think there are some improvements that could be
made. I'd like to discuss them in this ticket.
My proposal:
1. Transferring old results to the new matrices is a great idea, I
strongly support that.
2. In order to avoid confusion about "anonymous" results, we should
clearly separate new results from transfered results. Using a <ref>
comment is possible, but I'd like to do something more visible. We could
do something like {{result|fail|rc1|12345}} and then link rc1 to correct
wiki page (if {{result}} template is not suitable for it, we could create
{{oldresult}} template or similar). But even though it could be scripted,
it seems like too much work. Easier solution is to do
{{result|fail|previous run|12345}}, and if "previous run" is linked to
[[User:previous run]], we could use that wiki page for short explanation.
3. We should transfer only fails and warnings, not passes. The rationale
is that the pass result makes you (me, many people) think: "Hey, there's a
pass, let's work on something else". But pass from previous run doesn't
mean pass in current run (as I experienced today, when I changed two
previous passes to fails). Therefore warnings and fails are good to know
(we can check whether they are fixed or not, and we won't forget about
them), but passes are counter-productive - let's leave empty fields
instead.
4. All transfered results must have bugzilla number assigned. Otherwise
there's no information value. If I don't know what was broken, I can't
check whether it's been fixed or not.
5. We should document all of this in the "Test Results Format" table and
also advise people how to interact with these transfered results. My idea:
If there was a transfered warn/fail, and you checked that the problem is
fixed, remove it from the table (and put your result in there instead). If
you checked that the problem is still present, again remove it from the
table and put your result in there instead (referencing the same bug
number). If you didn't check the problem, put your result in the table and
leave the transfered result intact.
What do you think?
--
Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/175>
Fedora QA <http://fedorahosted.org/fedora-qa>
Fedora Quality Assurance
--
test mailing list
test@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test
