#35: Sync up install test priorities with the release criteria
--------------------------+-------------------------------------------------
Reporter: jlaska | Owner: jlaska
Type: task | Status: assigned
Priority: major | Milestone: Fedora 13
Component: Test Review | Version:
Resolution: | Keywords:
--------------------------+-------------------------------------------------
Comment (by rhe):
Replying to [comment:9 jlaska]:
> <skip>
>
> You make a good point. I keep going back and forth on ... if the use
cases aren't release critical, should we continue testing them? Anyone
have a suggested phrasing for tests we want to run, but they aren't
release critical?
>
Wow, I keep thinking this for a looong time but still can't find a good
solution. I think it depends on the goal for installation tests. We test
it whether to meet the release criteria or to fully check the
installation. In my view so far we design it for a full check, or we won't
create the repo cases. Then I think we can have extra label(s) for our
own. But it's really tough to think out one. ：）
> > Yes, PUD should be Alpha, I think.
>
> Okay, thanks. Updated wiki.
>
> > traceback debug mode is Alpha, isn't it? From Alpha requirements: "The
installer must be able to report failures to Bugzilla, with appropriate
information included"
>
> Maybe. As I read it, the criteria captures the test
[https://fedoraproject.org/wiki/QA:Testcase_Anaconda_save_traceback_to_bugzilla
QA:Testcase_Anaconda_save_traceback_to_bugzilla]. I feel that might need
to be expanded or a new criteria is added.
>
> How do folks feel about adding a criteria to the
[https://fedoraproject.org/wiki/Fedora_13_Beta_Release_Criteria
Fedora_13_Beta_Release_Criteria] ... something like ... ''The installer
must be able to copy failures logs to a remote system and offer a debug
shell for investigating failures, and support copying failure information
to a remote system.''
>
Agree.
> > updates.img working is probably Beta or Final, I'd think.
>
> Agreed. We currently have
[https://fedoraproject.org/wiki/QA:Testcase_Anaconda_updates.img_via_URL
QA:Testcase_Anaconda_updates.img_via_URL] listed as an Alpha priority
case. I think this is correct since the use case offers the only
mechanism to work around issues once the installer is released as Alpha.
Unfortunately, I can't see a Alpha (or Beta) release criteria to link this
back to. Any suggested updates to the criteria to accommodate this?
>
> With regards to
[https://fedoraproject.org/wiki/QA/TestCases/PackageSetsMinimalPackageInstall
QA/TestCases/PackageSetsMinimalPackageInstall]. Any concerns around
obsoleting this test?
Sometimes when sth wrong happens during default package installation,
minimal install is another choice. So I think we can just keep it.
--
Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/35#comment:11>
Fedora QA <http://fedorahosted.org/fedora-qa>
Fedora Quality Assurance
--
test mailing list
test@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test
