On Wed, 2018-05-30 at 14:23 +0200, Lukas Ruzicka wrote: > Hello, > > I was checking the release criteria and I spotted this one: > https://fedoraproject.org/wiki/Fedora_29_Final_Release_Criteria#Release_identification > which does not have a test case connected to it. However, there is one test > case that does pretty much the same, although uses a different attitude. It > is the > https://fedoraproject.org/wiki/QA:Testcase_base_edition_self_identification > > I think that the check for installed packages could be added to the above > mentioned test case and kill two birds with one stone: > > > 1. If using a specific release of Fedora, check that the *fedora-release* > package is installed: > - rpm -qi fedora-release > 2. Check that the *generic-release* package is in the repository: > - dnf search generic-release > > The added criteria to pass the test would be: > > 1. The fedora-release package is installed. > 2. The generic-release package is in the repository. > > What do you think about it? Unfortunately it's not quite sufficient. Just checking that a fedora- release package is installed does not enforce that criterion; there is pretty much always going to be a fedora-release package unless we really screwed up, but it may not contain the correct 'names, information and repository configuration'. There's two particular requirements we always have to remember to check: the release must be *at least* 1 (it cannot be 0.x, as this causes the 'Timbuktu warning' to appear in some installer modes), and the updates-testing repo *must* be disabled by default. Both those things are usually false until quite late in the release cycle; somewhere around Final freeze time releng will do a fedora-release package update that bumps the release to -1 and disables updates-testing. As for generic-release - doing a 'dnf search' from an installed system again doesn't quite cut the mustard, as that's looking in the 'fedora' repo, and until updates-testing is disabled, it's looking in the 'updates-testing' repo. At least conceivably, the package could be there but not in the actual release repository. It also may not be 'appropriately versioned' (again, the release should be -1 or higher). These have always been sort of magic things that we and releng magically know, and it's certainly true that we should write them into a test case to formalize the knowledge. I would not want to extend this test case to cover that, though. Lately I've actually been trying to make the test cases vaguely related to identification *more* specific and *less* general, in the interests of making them easier to automate. If we were going to extend an existing test case, https://fedoraproject.org/wiki/QA:Testcase_base_artwork_release_identification might be a better choice, but really, I think it would be best to have a new separate test case. One tricky thing is that we shouldn't mark it as a fail early in the cycle when fedora-release has u-t enabled and a release of < 1; this is correct at that time. The test case should cover this somehow - it should have two - or, in fact, three, thinking about Rawhide composes - different expected results for what the packages should be versioned and what repositories should be enabled at different points in the cycle. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx/message/I2OJIJDTQ75FG3M4TYL7UEHEF3F7AINP/
