On Tue, 2011-11-22 at 21:31 +0000, "Jóhann B. Guðmundsson" wrote: > On 11/22/2011 09:03 PM, Adam Williamson wrote: > > 2. Any update marked as 'critpath breaking' by a proven tester would be > > blocked from being pushed stable at all - automatically or manually - > > until the PT modified the feedback or it was overridden by someone with > > appropriately godlike powers (TBD, but probably not just the maintainer) > > Hum I thought the proven tester concept was being dropped anyway. At present that seems like where we're going, but I ignored it for the purposes of the Glorious Vision. The Glorious Vision does, I think, provide a reasonable suggestion as to how PTs could be useful if we had more detailed karma possibilities. > A) Have maintainers for critical path component provided test cases for > proven tester to use to test to differentiate them from fas-tester and > the "I installed this update and continued to use my system as normal > and did not note any regressions'"? In many cases no, but the nice thing about critpath is that in most cases we don't really need that. For some tricky components it'd be nice to have test cases that effectively explain exactly how that component intersects with critpath, but for almost all critpath components, I can't see that it would ever be wrong to hit the 'panic button' if you installed the update, rebooted, and the system blew up, for instance. The nice thing about a system based on *negative* feedback is that while it can be less than optimally efficient, it's almost always useful to at least some degree even when it's not perfect. > B) Has the scalability problem be solved as in the required minimum > amount of active proven testers has been met for this to actually work? See above: again the nice thing here is we don't _need_ a critical mass for a mechanism based on negative rather than positive results. Of course, the more testers we have the more issues we will likely identify, but a panic button is useful even if it only ever gets pressed once (as long as the press isn't a false one). Even if there were 99 other times when it _could_ have been pressed and wasn't, 1/100 success rate is better than 0/100. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
