> Hi folks! So at the meeting this week we agreed to add test cases for > N-1 upgrades. I've adjusted the templates used to generate the upgrade > test cases to allow for this, but now I notice there's kind of a naming > conflict. > > All the current upgrade test cases are named upgrade_dnf_previous_foo , > with 'previous' meaning 'upgrade from the previous release relative to > the one being tested'. But the FedoraVersion template that's used in > the upgrade test case templates also uses the term 'previous', only it > uses it to mean 'previous stable release'. In other words, right now > for the upgrade test case names 'previous' means 'Fedora 23', but for > the FedoraVersion template it means 'Fedora 22'. > > The way I implemented source release choice in the upgrade test case > template is to let you set a value that'll be passed through to the > FedoraVersion template. So we're in the kinda odd position right now > that our 'previous' test cases pass the value 'current' to > FedoraVersion, and to add N-1 test cases we'd have to name them > something like 'upgrade_dnf_previous2_foo' but have them pass the value > 'previous'. > > That seems like it'd be confusing for anyone who came along in a year > or two and wanted to figure out what was going on, so I'm proposing > that instead, we rename the existing upgrade tests to follow the > FedoraVersion template naming. We'd have 'upgrade_dnf_current_foo' test > cases which test upgrade from current stable to release under test (23 > -> 24 at present), and 'upgrade_dnf_previous_foo' test cases which > would test upgrade from previous stable to release under test (22 -> 24 > at present), and the names of the test cases would match the values > they use for the template params. Honestly, I've always found it confusing to refer to last stable release as "previous" in the test case name. So I welcome this change, because I think it makes more sense and is more in line with our terminology. > > The only slight drawback I can see is that it'd lead to a slightly odd > effect in testcase_stats: as the *newly added* test cases would have > the same name as the *current* test cases currently have, > testcase_stats would group them together - so testcase_stats would show > a full series of 'upgrade_dnf_previous' results for F24, but in fact > the first few of those would be F23->F24 upgrade tests, while the rest > would be F22->F24 upgrade tests. This doesn't really seem like a big > problem, though, we're very early in the cycle and testcase_stats is > just an aid for testers, it's OK for it to have this kind of wrinkle. I think this is not important at this moment. Thanks. -- test mailing list test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe: http://lists.fedoraproject.org/admin/lists/test@xxxxxxxxxxxxxxxxxxxxxxx
