On Thu, 2011-08-25 at 14:35 -0600, Stephen John Smoogen wrote: > On Thu, Aug 25, 2011 at 14:25, Chris Lumens <clumens@xxxxxxxxxx> wrote: > >> kickstart is a very broad area; you can write extremely complex > >> kickstart files that do a lot of stuff. So broadly what we'd need to do > >> is define a subset of kickstart functionality that we expect to work, > >> and then possibly divide that up by release phase (so some stuff must > >> work by Beta, the rest by Final, for e.g.) > >> > >> anaconda devs following this list, do you have any existing expectations > >> as to what level of kickstart functionality ought to be in place for > >> releases, and when you think would be appropriate? > >> > >> So far it seems everyone more or less agrees that it should be possible > >> to do at least a basic unattended kickstart install by Beta. > > > > You're right, kickstart is incredibly broad. I don't think we could > > ever hope to come up with criteria to cover all of it. I guess the best > > we can do is define criteria in terms of something else we already have. > > I would go for the classical kickstart test for Alpha: > > Does it take a minimal kickstart and build a default system. The > minimal being the exact stuff that would be created if a person just > clicked through a release. Oh, I like that. That's very good. > For Beta > > Take these X broken kickstarts, does it bail at the appropriate places. > Take these Y working kickstarts, does it work. > Where Y is a set of items that can be tested on say a KVM or a > "default" desktop defined somewhere. I'm not so keen on that; it's a bit specific. One of my fetishes with the criteria is to keep them generic so they don't have to keep being changed all the time; I wouldn't want a criterion to rely on some specific kickstart file we keep lying around somewhere. > For Final > The above and some subset of obscure items that can be tested reliably > somewhere that development and QA can replicate. again, seems a bit arbitrary; what we really need is the kickstart functionality we actually believe has to work, not an arbitrary set of interesting bugs we happen to be able to test. a minimal answer to this question would be fairly valid, I guess. at least in the short term. sometimes working up criteria in response to real-world cases - i.e. someone finds a kickstart problem, files a bug, proposes it as blocker, we agree it should be one and work up a criterion in response to it - has proved a decent way of doing things in the past. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
