On 3/24/22 11:12, Adam Williamson wrote:
On Mon, 2022-03-21 at 14:19 +1100, Ian Laurie wrote:
On 3/21/22 12:44, Adam Williamson wrote:
On Sun, 2022-03-20 at 17:01 +0000, rawhide@xxxxxxxxxxxxxxxxx wrote:
According to the schedule [1], Fedora 36 Candidate Beta-1.3 is now
available for testing. Please help us complete all the validation
testing! For more information on release validation testing, see:
https://fedoraproject.org/wiki/QA:Release_validation_test_plan
Just to explain the multiple composes - 1.2 and 1.3 are actually
identical in the end. 1.2 got a gnome-shell build one older than it was
supposed to have. 1.3 was an attempt to fix that, but it didn't work.
Not sure if releng want to try a 1.4, but nirik says the fix should be
to untag the older gnome-shell from the "f36-compose" tag and run the
compose again.
Is there a way to migrate test results from the 1.2 pages to the 1.3
pages? I spent 2 days testing LOL, would be a shame to lose the info.
Hi Ian! So I took a look at your 1.2 results, thanks a lot for all the
detailed notes! As a general point, the desktop test cases were written
primarily considering the release-blocking desktops (GNOME and KDE). As
you noticed, they don't necessarily exactly fit all the non-blocking
desktops; these don't always have the same default configurations or
philosophies about how things should work.
If you want to, it would be great to make obvious edits to the test
cases to improve things here (e.g. add a note to install a PDF viewer
if there isn't one installed by default, or turn on auto-mounting for
desktops where it isn't on by default). I try to keep such notes quite
generic - e.g. I'd write "if your desktop does not have a PDF viewer
installed, install one from the package management tool" rather than
"on LXQt, run dnf install evince". I find they age a bit better that
way. For the auto-mounting case specifically, though, I'd probably
include a note that on GNOME and KDE it *is* expected to be turned on
by default, and it's a bug if it isn't.
If you think a test case could do with more major surgery, perhaps
write a draft and send it to the list for review. I'm sure several of
them could be improved for the non-blocking desktops.
It looks like, in a few cases, a test case just doesn't really apply to
a desktop - e.g. LXqt not including abrt-gui. In these cases the best
thing to do may just be to edit the matrix template -
https://fedoraproject.org/wiki/Template:Desktop_test_matrix - and grey
out the relevant cell. This is the convention for indicating "this test
is not applicable to this environment". There are several such cases
already from which you can copy the syntax.
If you think there are cases where a desktop doesn't include an app for
some purpose but really *should*, it'd be appropriate to file those as
bugs. If the spin has a pseudo-component in Bugzilla, use that,
otherwise, I'd file it against some package that's a core part of the
desktop and maintained by the person who seems to look after the
desktop in general.
As for transferring the results, we can do that for sure, but for non-
blocking test cases it isn't actually super important. We use
testcase_stats a lot to see the whole story of when things were tested
and what the results were throughout the cycle, not just for the most
recent compose, and your results are certainly captured there:
https://openqa.fedoraproject.org/testcase_stats/36/Desktop.html
and they will also be counted in the "heroes of Fedora" stats at the
end of the cycle, these count all tests anyone runs against any compose
validation event, it doesn't matter if it was the one that got released
or not. You filed bugs for the issues that were clear bugs, as well.
I'll transfer over the results just for completeness, but just wanted
to note it isn't actually super important to do so :) It's mainly
important for blocking results, for when we do a glance over the
results for *just* the compose we're planning to ship at the Go/No-Go
meeting to check it's been fully tested. So I don't usually bother
transferring non-blocking results.
Thanks again for the great testing!
P.S. One more thing - you should usually never need to bother running
the sizecheck test as we have that automated, coconut does it. It's
only necessary to run that if you don't see results from coconut for
some reason.
Thanks for that information. My inclination would be to send any
proposed edit to the list for review. I guess an attachment of a copy
of the the content of the edit window that someone could plonk in and
preview would be the way to do it? Anyway I'd certainly like to
contribute some tweaks before F37.
Ian
--
Ian Laurie
FAS: nixuser | IRC: nixuser
TZ: Australia/Sydney
_______________________________________________
test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
