#154: Tracker: critical path test case creation
--------------------------+-------------------------------------------------
Reporter: adamwill | Owner: adamwill
Type: enhancement | Status: new
Priority: major | Milestone: Fedora 15
Component: Wiki | Version:
Resolution: | Keywords:
--------------------------+-------------------------------------------------
Comment (by adamwill):
sigh, I keep getting ready to send this out to lists and then hitting one
more snag. =)
so, in discussion with jlaska, we both think it makes most sense to only
create two new types of category: one for associating test cases with
specific packages, one for marking them as critical path test cases.
Critical path test cases should be in the category
Critical_path_Test_Cases . Note that this means *any test case which is
primarily concerned with verifying critical path test cases* should be in
this category. It doesn't mean any test case associated with a critical
path package should be in the category, because critical path packages can
have non-critical path uses and hence non-critical path test cases: a test
case to test, say, tape drive functionality in the kernel is associated
with a critical path package (the kernel) but is not a critical path test.
The category name for specific source packages should be:
Package_(srcrpmname)_Test_Cases
same reasoning as my initial proposal for the test case name: we need to
specify that (srcrpmname), in this context, is specifically a source RPM
name. I can imagine, for instance, we might want to create a generic
category for Python test packages, called Category:Python_Test_Cases, and
put test cases into that category which are not actually test cases for
the Python .src.rpm. So we'd also have a
Category:Package_python_Test_Cases . That could probably be a member of
Category:Python_Test_Cases , but not vice versa.
I think the page name of each test case can be a bit more flexible and we
don't really need to worry about defining it, the categories are the
crucial thing.
Do we also want to suggest a category for each .src.rpm along the lines
of:
Category:Package_python_core_Test_Cases
? Rationale: I'm thinking about how we want the bodhi or fedora-easy-karma
listing for each package to look. I'm thinking of something like:
-------------------------------------------------------
Update: python-2.7.9-blahblah
This update does blah blah foo foo whee whee poop.
The following test cases can be used to check the functionality of this
package.
'''Critical path tests'''
These tests verify critical path functionality (link to critical path
process page) and should take highest priority:
* links to all tests in Category:Package_python_Test_Cases *and*
Category:Critical_path_Test_Cases
'''Core functionality tests'''
These tests verify core functionality of the package and should take
highest priority after critical path tests:
* links to all tests in Category:Package_python_core_Test_Cases (but not
Category:Critical_path_Test_Cases)
'''Remaining tests'''
These tests verify non-critical path, non-core functionality:
* links to all other tests (in Category:Package_python_Test_Cases but not
Category:Package_python_core_Test_Cases or
Category:Critical_path_Test_Cases)
------------------------------------------------------
------------------------------------------------------
The thinking being that some packages will grow fairly large sets of tests
(abrt already has, for instance) and it would help to isolate core
functionality tests for rapid sanity checking.
random side observation: our naming of test case pages and categories
currently stinks.
next big concern: where do we effectively document this? All we currently
have on test cases is:
https://fedoraproject.org/wiki/Category:Test_Cases
I can add some overview and stuff to that page but it's not a terribly
visible page, really. I guess the best approach is to edit that page and
add some links to it from strategic other places, especially developer
instruction pages. Thoughts?
--
Ticket URL: <https://fedorahosted.org/fedora-qa/ticket/154#comment:16>
Fedora QA <http://fedorahosted.org/fedora-qa>
Fedora Quality Assurance
--
test mailing list
test@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe:
https://admin.fedoraproject.org/mailman/listinfo/test
