On Tue, Jul 12, 2016 at 11:38:01AM -0700, Adam Williamson wrote: > This isn't really correct, because there is no simple relationship > between 'bugs claimed to be fixed actually are fixed' and 'update > should be released'. Both of these are possible: > > 1) an update which fixes the bugs it claims to fix, but should *NOT* be > released > 2) an update which does not fix all the bugs it claims to fix, but > *SHOULD* be released > > An example of 1) is an update which claims to fix a minor bug, and > does, but creates a *major* bug. e.g., fixes a typo in the package > description, but causes the app not to run at all. This update should > be given -1 karma (negative response to 'Is the update generally > functional?'), not +1. > > An example of 2) is an update which claims to provides a critical > security fix and a trivial bug fix, and *does* fix the security issue, > but the trivial bug fix doesn't work. This update should be given +1 > karma (positive response to 'Is the update generally functional?'), not > -1. Sure, and I said nothing to contradict that. A good test involves verification of fixes *and* regression tests, not one or the other. However using lack of regression tests as an excuse for not verifying fixes (and more importantly, still leaving positive feedback) is not acceptable. The points you mention go into intricacies of testing feedback whereas I am talking about the very basics. > This is in fact *why*, in Bodhi 2.0, there are separate feedback > entries for each individual bug listed by the update - so testers can > separate 'does or does not fix bug X' feedback from 'update generally > works' feedback. > > It is possible (in my opinion) for a tester to reasonably provide 'Is > the update generally functional?' feedback without actually checking > all or even any of the claimed bug fixes, and I've done this myself > quite often. It can quite often be difficult (if the bug is in a very > complex use case) or impossible (if the bug is specific to e.g. a piece > of hardware you don't own) to check bug fixes. Sure, such feedback is very useful, but I don't think it should be accompanied by a karma +1. The 'update generally functional' should only have options of karma 0 and -1 IMO. Siddhesh -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://lists.fedoraproject.org/admin/lists/test@xxxxxxxxxxxxxxxxxxxxxxx
