On Mon, 2022-03-21 at 14:19 +1100, Ian Laurie wrote: > On 3/21/22 12:44, Adam Williamson wrote: > > On Sun, 2022-03-20 at 17:01 +0000, rawhide@xxxxxxxxxxxxxxxxx wrote: > > > According to the schedule [1], Fedora 36 Candidate Beta-1.3 is now > > > available for testing. Please help us complete all the validation > > > testing! For more information on release validation testing, see: > > > https://fedoraproject.org/wiki/QA:Release_validation_test_plan > > Just to explain the multiple composes - 1.2 and 1.3 are actually > > identical in the end. 1.2 got a gnome-shell build one older than it was > > supposed to have. 1.3 was an attempt to fix that, but it didn't work. > > Not sure if releng want to try a 1.4, but nirik says the fix should be > > to untag the older gnome-shell from the "f36-compose" tag and run the > > compose again. > > Is there a way to migrate test results from the 1.2 pages to the 1.3 > pages? I spent 2 days testing LOL, would be a shame to lose the info. Hi Ian! So I took a look at your 1.2 results, thanks a lot for all the detailed notes! As a general point, the desktop test cases were written primarily considering the release-blocking desktops (GNOME and KDE). As you noticed, they don't necessarily exactly fit all the non-blocking desktops; these don't always have the same default configurations or philosophies about how things should work. If you want to, it would be great to make obvious edits to the test cases to improve things here (e.g. add a note to install a PDF viewer if there isn't one installed by default, or turn on auto-mounting for desktops where it isn't on by default). I try to keep such notes quite generic - e.g. I'd write "if your desktop does not have a PDF viewer installed, install one from the package management tool" rather than "on LXQt, run dnf install evince". I find they age a bit better that way. For the auto-mounting case specifically, though, I'd probably include a note that on GNOME and KDE it *is* expected to be turned on by default, and it's a bug if it isn't. If you think a test case could do with more major surgery, perhaps write a draft and send it to the list for review. I'm sure several of them could be improved for the non-blocking desktops. It looks like, in a few cases, a test case just doesn't really apply to a desktop - e.g. LXqt not including abrt-gui. In these cases the best thing to do may just be to edit the matrix template - https://fedoraproject.org/wiki/Template:Desktop_test_matrix - and grey out the relevant cell. This is the convention for indicating "this test is not applicable to this environment". There are several such cases already from which you can copy the syntax. If you think there are cases where a desktop doesn't include an app for some purpose but really *should*, it'd be appropriate to file those as bugs. If the spin has a pseudo-component in Bugzilla, use that, otherwise, I'd file it against some package that's a core part of the desktop and maintained by the person who seems to look after the desktop in general. As for transferring the results, we can do that for sure, but for non- blocking test cases it isn't actually super important. We use testcase_stats a lot to see the whole story of when things were tested and what the results were throughout the cycle, not just for the most recent compose, and your results are certainly captured there: https://openqa.fedoraproject.org/testcase_stats/36/Desktop.html and they will also be counted in the "heroes of Fedora" stats at the end of the cycle, these count all tests anyone runs against any compose validation event, it doesn't matter if it was the one that got released or not. You filed bugs for the issues that were clear bugs, as well. I'll transfer over the results just for completeness, but just wanted to note it isn't actually super important to do so :) It's mainly important for blocking results, for when we do a glance over the results for *just* the compose we're planning to ship at the Go/No-Go meeting to check it's been fully tested. So I don't usually bother transferring non-blocking results. Thanks again for the great testing! P.S. One more thing - you should usually never need to bother running the sizecheck test as we have that automated, coconut does it. It's only necessary to run that if you don't see results from coconut for some reason. -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net _______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
