#151: Tests consistent with Criterion
--------------------------+-------------------------------------------------
Reporter: rhe | Owner: rhe
Type: enhancement | Status: new
Priority: major | Milestone: Fedora 16
Component: Wiki | Version:
Resolution: | Keywords: retrospective
--------------------------+-------------------------------------------------
Comment (by adamwill):
I have done a survey of the alpha criteria, checking for associated tests.
Here are the results. 'partial' indicates that relevant tests are present
but do not entirely cover the criterion's requirements. 'not-in-matrix'
indicates a test case is written but not included in the current test
matrices. 'NO TEST' indicates that, as far as I could find, there is no
test case written for the criterion. Following these keywords, I have
listed the relevant test cases for each criterion (where applicable).
We can open subsidiary tickets to this bug for each specific issue
identified here.
There must be no file conflicts (cases where the files in some packages
conflict but the packages have explicit Conflicts: tags are acceptable) or
unresolved package dependencies during a media-based (CD/DVD) install
https://fedoraproject.org/wiki/QA:Testcase_Mediakit_FileConflicts
All dedicated installer images must boot to the graphical boot menu and
allow the user to select install options. If no option is selected, the
installer should load after a reasonable timeout
partial
https://fedoraproject.org/wiki/QA/TestCases/BootMethodsBootIso
https://fedoraproject.org/wiki/QA/TestCases/BootMethodsDvd
https://fedoraproject.org/wiki/QA:Testcase_Boot_Methods_efidisk.img
The installer must boot (if appropriate) and run on all primary
architectures from default live image, DVD, and boot.iso install media
not-in-matrix
https://fedoraproject.org/wiki/QA:Testcase_Live_Image_Boot
https://fedoraproject.org/wiki/QA/TestCases/BootMethodsBootIso
https://fedoraproject.org/wiki/QA/TestCases/BootMethodsDvd
The graphical boot menu for all installation images should include an
entry which causes both installation and the installed system to use a
generic, highly compatible video driver (such as 'vesa'). This mechanism
should work correctly, launching the installer and attempting to use the
generic driver
NO TEST
The installer must be able to use at least one of the HTTP or FTP remote
package source options
https://fedoraproject.org/wiki/QA:Testcase_Http_Repository (redundant with
ftp_repository?)
https://fedoraproject.org/wiki/QA:Testcase_Additional_Http_Repository
https://fedoraproject.org/wiki/QA:Testcase_Additional_Ftp_Repository
https://fedoraproject.org/wiki/QA:Testcase_Ftp_Repository
The installer must be able to use the DVD local package source options
https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd
The installer must be able to complete an installation using the text,
graphical and VNC installation interfaces
https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Text
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_VNC
The installer must be able to complete package installation with the
default package set for each supported installation method
https://fedoraproject.org/wiki/QA/TestCases/InstallSourceDvd
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Text
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_VNC
The installer must be able to complete an installation using IDE, SATA and
SCSI storage devices, with the default file system and LVM
https://fedoraproject.org/wiki/QA/TestCases/PartitioningExt4OnNativeDevice
https://fedoraproject.org/wiki/QA:Testcase_install_to_SATA_device
https://fedoraproject.org/wiki/QA:Testcase_install_to_SCSI_device
https://fedoraproject.org/wiki/QA:Testcase_install_to_PATA_device
The installer must be able to complete an installation using the entire
disk, existing free space, or existing Linux partitions methods, with or
without encryption enabled
partial
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_install
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_%28use_free_space%29_install
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_%28encrypted%29_install
(existing partitions method does not seem to be tested)
The rescue mode of the installer must start successfully and be able to
detect and mount an existing default installation
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_rescue_mode
The installer must be able to report failures to Bugzilla, with
appropriate information included
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_save_traceback_to_bugzilla
In most cases (see Blocker_Bug_FAQ), a system installed according to any
of the above criteria (or the appropriate Beta or Final criteria, when
applying this criterion to those releases) must boot to the 'firstboot'
utility on the first boot after installation, without unintended user
intervention. This includes correctly accessing any encrypted partitions
when the correct passphrase is supplied. The firstboot utility must be
able to create a working user account
NO TEST
Following on from the previous criterion, after firstboot is completed and
on subsequent boots, a system installed according to any of the above
criteria (or the appropriate Beta or Final criteria, when applying this
criterion to those releases) must boot to a working graphical environment
without unintended user intervention. This includes correctly accessing
any encrypted partitions when the correct passphrase is supplied
NO TEST (partly implied by more sophisticated desktop tests)
When booting a system installed without a graphical environment, or when
using a correct configuration setting to cause an installed system to boot
in non-graphical mode, the system should boot to a state where it is
possible to log in through at least one of the default virtual consoles
partial
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Text
It must be possible to run the default web browser and a terminal
application from all release-blocking desktop environments. The web
browser must be able to download files, load extensions, and log into FAS
https://fedoraproject.org/wiki/QA:Testcase_desktop_browser
The installed system must be able to download and install updates with yum
and the default graphical package manager in all release-blocking desktops
https://fedoraproject.org/wiki/QA:Testcase_desktop_updates
The default Fedora artwork must either refer to the current Fedora release
under development (Fedora 16), or reference an interim release milestone
(e.g. Alpha or Beta). If a release version number is used, it must match
the current Fedora release under development. This includes artwork used
in the installer, firstboot, graphical boot, graphical login and desktop
background.
NO TEST
A system logging infrastructure must be available and enabled by default.
It must provide at least basic local file-based logging of kernel
messages, and allow other components to write log messages. This must be
done in accordance with relevant standards accepted by the Project
NO TEST
--
Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/151#comment:3>
Fedora QA <http://fedorahosted.org/fedora-qa>
Fedora Quality Assurance
--
test mailing list
test@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test
