> (and looking at> https://fedoraproject.org/wiki/Test_Results:Current_Desktop_Test , it seems
> we don't even have one, because what we usually do is to thoughtlessly
> perform some of those extra graphical steps, like opening an application,
> on top of the vague "package install/remove" test case - which is certainly
> something that can be improved in the description of one of those places).
>
We do have the test case for Core Applications and the ones for Browser
and Terminal. Software even gets a tryout for updating. Question is: Is
this sufficient?
https://fedoraproject.org/wiki/QA:Testcase_workstation_core_applications is just about a few apps being present, nothing more. I edited the test case to highlight that. But I was wrong, we have https://fedoraproject.org/wiki/QA:Testcase_desktop_menus which checks the basic functionality of all applications, including gnome-software. Which includes exactly installing/removing an app, running the app directly from the software center, and a couple of other actions, probably. We could have a separate test case for high profile applications like gnome-software, which would even help us define what "basic functionality' means (if it's in the test case, it's likely basic). Is that necessary? It probably isn't. Should we have a separate test case? Shrug, I don't have clear preferences.
I commented here a couple of times, a year or so ago, on testing all the
applications that get installed as part of the Fedora install. I believe
there was a test case along those lines then. I think what happened was
that it was viewed as a lot of work and as I recall that's where the
core applications test case came from.
No, see the links above and read the test cases carefully. The "all apps must work" testcase is still here, and "core workstation apps must be present" testcase is just about being present. Yes, QA dislike the former testcase and criterion, it's vague and a lot of work. We will want to do something about it and I should have sent a proposal directly related to this some time ago. I still haven't because I was waiting on clearing up some situation regarding new blocking artifacts in F32, but anyway, you'll some new proposal from me soon.
Back then the original test was
to Open, check the About, and Close each of the "standard" applications.
Well, the instruction to check the About menu is still there in the test case :)
I do this now as part of my "as deployed" testing after I run the
standard test cases. Some of the "standard" applications do not get
tested since they are removed in the "as deployed" configuration.
I think all the standard applications should get a basic dead or alive
test. This might be able to be limited somewhat for things like the
LibreOffice suite since there are so many common components for the
different LibreOffice applications.
If "standard apps" means apps installed by default, that's exactly what the "desktop menus" test case is about. (And I guess we should rename it to make it clearer).
It might also be be beneficial to install a "non-standard" application
and not only verify that the package manager worked, but also give the
installed application a dead or alive test.
That's definitely beneficial, please do it often :-) But we're not likely to block on a broken application that's not pre-installed. So that's why it's not among test cases - we design test cases around release criteria.
_______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx
