On Mon, 2014-01-27 at 19:24 -0800, Adam Williamson wrote: > On Tue, 2014-01-28 at 10:39 +0800, Mathieu Bridon wrote: > > On Mon, 2014-01-27 at 15:11 -0700, Kevin Fenzi wrote: > > > On Mon, 27 Jan 2014 10:18:56 -0500 > > > Matthew Miller <mattdm@xxxxxxxxxxxxxxxxx> wrote: > > > > > > ....snip.... > > > > > > > * possibly adding a "what should users test?" field to the update > > > > info. > > > > > > > > I know that there's a "notes" field in the update, but maybe it'd > > > > help to explicitly include testing instructions? > > > > > > > > Each package in the pkgdb (or in git, or wherever) could have > > > > a standard list included in each update as the default (for > > > > example, for 'calc', it might be to try `calc -q > > > > read /usr/share/calc/regress.cal`. That would duplicate a likely > > > > smoke-test, though, so maybe also "run interactively and make sure > > > > basic math works". > > > > > > > Then, each update could also optionally (and this would be > > > > presented in bold if it were used) say something like "New release > > > > adds log() function; please test that it works", or "Severe bug where > > > > 1+1=3 corrected; please test that the answer now corresponds with > > > > consensus reality." > > > > > > I could have sworn we already had something like this where bodhi would > > > add a link to a wiki page for test plan on a package if that wiki page > > > existed. I can't seem to find it now, so perhaps it was just something > > > we talked about, but never implemented. > > > > Nope, you're right. :) > > > > https://github.com/fedora-infra/bodhi/blob/develop/bodhi/model.py#L191 > > > > For example, the test case pages for the package 'foo' need to be added > > to the 'Category:Package foo test cases' category. > > > > There's also an option in the config file which must be switched on for > > this to work: 'query_wiki_test_cases' > > > > And here is an update where this is, in fact, actually used: > > https://admin.fedoraproject.org/updates/FEDORA-2014-1465/ > > This is the package test plan thing - the QA docs on it are at > https://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation . I > usually use xorg-x11-drv-nouveau updates as a handy example of it in > action. You can create a test case for a package and add it to the > category Category:Package_(packagename)_test_cases (where 'packagename' > is the .src.rpm name), and it will show up in Bodhi like this. Actually, if it wouldn't be _too_ much work, a handy extension to this suggests itself: it might be nice to be able to use a similar mechanism to mark a test case as being relevant to an entire group of packages - whether that means 'any comps group' or we just special-case critpath, or whatever. It'd be nice to write a 'install the update, then try installing and removing a package' test case, and mark it to show up on all critpath updates, maybe. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
