On Mon, 2013-01-14 at 14:25 +0100, Michael Schwendt wrote: > Has anyone ever before retrieved and refined statistics from bugzilla? Well, sure. Technically it's not difficult to do. What can be quite difficult is designing a query that actually makes sense and gives you useful data. What would we want to take out of Bugzilla in this case? All bugs filed against Fedora 18? All bugs filed against anaconda? Bugs that got fixed? Bugs that were blockers or NTH? Which of those is a 'useful' data set? It's a bit of a rabbit hole. We had an effort for a while to try and draw some long-term statistics from Bugzilla on an ongoing basis, related to bug triage work. That was a pretty complex project and never quite entirely got off the ground. This thread in general is a pretty good example of why it's very tricky to try and 'statistically' list out and thank contributors to F/OSS projects because there's just so many forms of contribution and it's practically impossible to nail them all down into numbers. I'm always hesitant to do this kind of thing for that reason. But I still applaud Kamil's effort and I think before anyone worries that it's undervaluing some contributions and overvaluing others, please just remember that context: any such effort is necessarily going to be incomplete, so relax and take it with a pinch of salt! No-one is being graded on their performance, it was just an incomplete attempt to say thanks to at least *some* of the people who've worked so hard on F18 validation. > > The wiki was only supposed to be a a short term solution because we > > never found a testing system to that quite suited our needs. > > It demands special efforts of the people who act in that area. In similar > ways, Fedora Packaging requires special efforts of people who do packages. > Hurdles everywhere. ;) Well, I mean, just about any kind of 'planned testing' is going to require 'special effort' of some kind. Fundamentally, we want to do specific testing of specific composes. That means we need in some way to denote 'this test is part of the validation testing for this compose'. The production and communication of that information cannot possibly be free. It is going to 'cost' someone, somewhere, some labor, at some point. No whizzy improved TCMS will change that. > I once had a look at what it would take to contribute "an official" test > of Fedora Test Release media (e.g. my pet peeve scenario "DVD image on HDD"), > but the Wiki pages were like a maze, very brief or for insiders only, new > images were thrown out quickly (with no rsync access), with hardly any > activity on test-list but probably heavy activity only on IRC. As I wrote, > not everyone's cup of tea. There's only a limited amount we can do about any of that. Again, we're working with fundamental constraints: we have a very short time to do image validation and a lot of validation to do (and, usually, a lot of bugs to fix). The fast revision cycle of TC/RCs, the limited download options, and the amount of test cases are all basically unavoidable given the release cycle we're operating within. I find it odd that you'd find the Wiki pages to be 'like a maze', though - there are simply three primary pages for each TC/RC and all three are directly linked to in the release announcement for each TC/RC. It's really not any more complicated than that. Each TC/RC has Install, Base and Desktop pages, with a bunch of test cases on each one. It doesn't seem like a hugely complex setup and I don't think it can really be made any simpler within the Wiki context. The current setup works better for the workflow 'I want to help test this build, let's go and see what tests need doing' than 'I want to do this specific test on the current build, I'll do it and then figure out where to put my result', admittedly. If anyone has ideas for improving that it'd be great, but again I'm not sure it can be made too much better within the Wiki and possibly it couldn't be improved even if we replaced the wiki - even in a 'better system' we'd have several dozen test cases and I'm not sure how much it's possible to 'automate' this particular flow, where a tester has a specific test they like to do, but doesn't know whether that test is one of the 'official' validation tests and, if it is, which one it is. That seems like a rather difficult thing to do intrinsically, to me. Anyhow, for the record, I can solve this particular one for you with the magic of the human brain: the closest 'official' test case to what you're talking about is https://fedoraproject.org/wiki/QA:Testcase_install_repository_Hard_drive_variation , and it's part of the 'General Tests' matrix on https://fedoraproject.org/wiki/Test_Results:Current_Installation_Test . We probably need to resurrect the 'graphical' version of that test case, since newUI does let you graphically select an ISO as an install source now. > If community people take the time to become familiar with all that, > that's great. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
