> > We have this: > > https://fedoraproject.org/wiki/QA:Testcase_Package_Sets_Default_Package_Install > > > > That covers GNOME. We could modify it and ask the tester to pick > > any > > one of the release blocking desktops to be installed. Do we want to > > make it fuzzy? (You won't know exactly what was tested). Or we can > > add > > a new KDE test case. Or we can leave it as it is. In this case, all > > variants seem OK to me. I would probably make it fuzzy or do > > nothing. > > I think Josef's probably right here, the most straightforward thing > to > do would seem to be to just test it explicitly. It's in the criteria, > writing the test case(s) should be simple, and they're not onerous > tests > to do, so I don't really see a problem. > > I think 'the default package set install works' is a Thing in itself, > so > I'd probably be more in favour of adding a separate test case which > covers installing each of the release-blocking desktops and can be > marked as 'pass' only if they all work. In practice, whoever's doing > the > test obviously can 'count' the default_package_install result towards > the release_blocking_package_sets_install (or whatever we call it) > test > case if the default package set at the time happens to correspond to > one > of the release-blocking desktops, as it does now. > > So I'd imagine this would happen: > > 1. Tester checks that a default install works, marks > default_package_install test as pass > 2. Tester checks that a KDE install works, and then marks > release_blocking_package_sets_install as pass > > in the future if our default package set is somehow not a release > blocking desktop any more, or we get more release blocking desktops, > everything continues to work out, testers just have a bit more > testing > to do before marking release_blocking_package_sets_install as pass. That is algorithmically perfect, but over-engineered [1]. I think it's much easier to have a default_package_install and a KDE_package_install and adjust it if the defaults changes, than to think about the correct workflow every time we execute the tests (I understand I'll be fired for saying this, but I tend to forget these things. I also want to make release validation more accessible for newcomers). [1] $ python -c 'import this' -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: https://admin.fedoraproject.org/mailman/listinfo/test
