On Tue, 2014-01-28 at 15:19 -0500, Matthew Miller wrote: > On Tue, Jan 28, 2014 at 10:39:32AM +0800, Mathieu Bridon wrote: > > > I could have sworn we already had something like this where bodhi would > > > add a link to a wiki page for test plan on a package if that wiki page > > > existed. I can't seem to find it now, so perhaps it was just something > > > we talked about, but never implemented. > > Nope, you're right. :) > > https://github.com/fedora-infra/bodhi/blob/develop/bodhi/model.py#L191 > > For example, the test case pages for the package 'foo' need to be added > > to the 'Category:Package foo test cases' category. > > Huh, lookit that. That's great! So I guess my suggestion changes to "make > that more visible". > > Also my head is exploding a little bit, because it's yet another case of the > wiki doing a lot of different types of work. That's because I came up with the design, and I'm an idiot monkey who knows how wikis work and not much else. :P Also, tbf, it's because we use the wiki as our TCMS (test case management system - look, I'm using jargon to pretend I have some kind of formal knowledge about my job!), and as long as we're doing that, it's probably no *more* hideous to use wiki categories for this kind of purpose as it is to weld on some kind of external method for categorizing test cases in this way. Using the wiki as a TCMS is of course a gross hack, but it comes with a surprising number of advantages - see https://fedoraproject.org/wiki/Tcms_Comparison . It's very https://xkcd.com/1172/ , but goddamnit, my spacebar heating works. We're not opposed to moving to some better in principle, but it would be a moderate amount of work and we do at least want the 'better' thing to be really and truly and concretely better and not just 'at least it's actually called a TCMS' :P > I wonder if it would help both > of these things to move the test plans to live in package git (right next to > the future taskotron scripts? Well, what would they look like, and how would you read them? What we're calling a 'test plan' for these purposes is basically just a category. Sure, we could stick some kind of metadata in some format in package git that just included the list of test cases associated with that package, but that doesn't seem like a fantastic design either, and it comes with the rather substantial downside that you now need commit rights to a package in order to write a test plan for it (or the co-operation of someone who has those rights). Of the relatively few test plans we have so far, most were written by QA folks, not package maintainers. > And, while clearly helpful, I think it suffers from a little bit of tl;dr. > Maybe we could embed summaries of each right into the Bodhi page? I'm not quite clear what you mean here - what's tl? Summaries of what? The *user experience* here is indeed that you just see the test cases right in the Bodhi page (or in whatever other interface you're using to view the update - easy-karma, gooey-karma, whatever). The documents describing how the system works and how to create test plans etc are somewhat long-winded, yeah, in the latter case because I wrote it. :P Are you saying you'd like a summary of how to create a test plan right in the Bodhi page for an update? that seems a bit inappropriate. > The other thing is that I think that in addition to the per-package test > cases, it would be good to encourage updates to include at least a line > about what is of particular interest to test in _this_ release. Sometimes, > that's the same as the update Details, but not always. It would indeed be nice if maintainers did this, though of course it's not relevant to the case we're currently considering (as the broken content in the update was a mistake and the maintainer didn't even know it was there, he'd hardly have been likely to highlight it for testing). -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
