On Tue, 2010-12-21 at 15:53 -0700, Kevin Fenzi wrote: +AD4 On Tue, 21 Dec 2010 17:11:15 +-0000 +AD4 Adam Williamson +ADw-awilliam+AEA-redhat.com+AD4 wrote: +AD4 +AD4 +AD4 Hi, everyone. So, in the recent debate about the update process it +AD4 +AD4 again became clear that we were lacking a good process for providing +AD4 +AD4 package-specific test instructions, and particularly specific +AD4 +AD4 instructions for testing critical path functions. +AD4 +AD4 ...snip... +AD4 +AD4 +AD4 Comments, suggestions and rotten fruit welcome :) I'm particularly +AD4 +AD4 interested in feedback from package maintainers and QA contributors in +AD4 +AD4 whether you feel, just after reading these pages, that you'd be +AD4 +AD4 confident in going ahead and creating some test cases, or if there's +AD4 +AD4 stuff that's scary or badly explained or that you feel like something +AD4 +AD4 is missing and you wouldn't know where to start, etc. +AD4 +AD4 ...snip... +AD4 +AD4 Great work Adam and QA folks. +ADs) +AD4 +AD4 From a quick glance over this looks good to me, and I would be happy to +AD4 start trying to make test cases. A few random things: +AD4 +AD4 +ACo Would it be worth noting that anyone can make a test case, it doesn't +AD4 need to be the maintainer, right? Oh that raises another good point. Unlike the Test+AF8-Results: wiki name space used for Test Days and similar test events, the QA: namespace does not allow anonymous edits. +AD4 +ACo Also, might be worth noting that if you run into a specific bug as a +AD4 maintainer and fix it, thats a great time to go add a test case to +AD4 specifically test that (since it's fresh in your mind). Great point+ACE I've also seen folks use placeholders in bugs when triaging or fixing them. For example, there are bugzilla keywords that can be used to tag bugs for future test case review, such as 'NeedsTestCase' (see https://bugzilla.redhat.com/describekeywords.cgi). +AD4 +ACo finally, it's ok to just start in on some tests and add them over +AD4 time, right? We don't want to care if a package has incomplete +AD4 coverage right off the bat right? +AD4 +AD4 +AD4 +AD4 +AD4 it also mentions one big current omission: dependencies. For instance, +AD4 +AD4 it would be very useful to be able to express 'when yum is updated, we +AD4 +AD4 should also run the PackageKit test plan' (because it's possible that +AD4 +AD4 a change in yum could be fine 'within itself', and all the yum test +AD4 +AD4 cases pass, but could break PackageKit). That's rather complex, +AD4 +AD4 though, especially with a Wiki-based system. If anyone has any bright +AD4 +AD4 ideas on how to achieve this, do chip in+ACE Thanks. +AD4 +AD4 Yeah, tough one. Not sure how best to handle that. Perhaps just a +AD4 'Dependencies:' type header asking you to make sure you test dependent +AD4 packages and see their test cases? It occurred to me that transclusion might work in a pinch (http://www.mediawiki.org/wiki/Help:Transclusion). It can be non-obvious, but it might allow us to include PackageKit test cases in the yum test case category by doing +AHsAew-Category:Package+AF8-PackageKit+AF8-test+AF8-cases+AH0AfQ. I've never tried this with Categories, so not sure if this works. I'll have to test this out. +AD4 Thanks again for working on this+ACE Agreed+ACE Great to see this coming together. Thanks, James
Attachment:
signature.asc
Description: This is a digitally signed message part
-- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel
