On Wed, Nov 7, 2018 at 11:35 AM Lukas Ruzicka <lruzicka@xxxxxxxxxx> wrote:
Hello Fedora QA and friends,
I would like to propose a change of what we define as core applications because I feel that how it is done today is not sufficient. Let me explain.
## How it works today?
The *core applications* are defined in this testcase (https://fedoraproject.org/wiki/Workstation/Technical_Specification#Core_Applications).
The correct link to the test case:
In our matrices, the core applications are only tested for Gnome Workstation.
Actually, this might be a misunderstanding. The testcase is called *Workstation* core apps, and the technical specification wiki is in the *Workstation* namespace. The Workstation SIG have defined their core apps, and we have a test case for them. There are no other core apps. So sure, workstation core apps are tested on workstation images, and nowhere else, because that wouldn't make sense :)
The appropriate test case (https://fedoraproject.org/wiki/Workstation/Technical_Specification#Core_Applications) requires that all core applications are installed on the system and that they start. The functionality is apparently tested by the *Desktop Menus Testcase*.
I think we should adjust the test case and ask the tester to verify that the apps are present. Because that's the idea behind it, core apps must be present (i.e. they must not fall out of the image by accident). Their startup and functionality is tested by another test case [1], that's correct, so it's an unneeded duplication to also require it in this test case. Once the test case is redefined to only check for presence (say rpm and a desktop icon), it can then be easily automated.
## How I think it could work?
I think we could change how we approach the core applications, how we test them and where we test them. For example:
* A list of generic core applications should be made (or the old list used, see above), so that core applications are not limited to a certain desktop environment (although we only test Gnome). By *generic* I mean that we should not explicitly say, if the terminal application is *gnome-terminal* or anything else. We only say it is a *terminal* application and each spin will pick up what suits best for them.
Well, the test cases mostly match release criteria, the release criteria only apply to blocking images, and blocking *desktop* images contain GNOME and KDE on x86_64 and XFCE on arm. So even if the test case says "The environment MUST contain a terminal, a web browser and a text editor", what does the MUST mean for e.g. LXDE? Nothing really. Also, I expect the intersection of core apps across various environments to be quite small (terminal, web browser, text editor, file manager) and pretty much obvious. If you include Server and Cloud in the mix (and they should not be left behind, if these are generic requirements), the intersection might be empty. So instead of generic requirements, per-SIG requirements might be a better fit. We already have Workstation core apps, we can define KDE core apps, XFCE core apps, Server core apps (which we somewhat already have, just in a different structure, in our existing criteria), ...
* Core applications should be promoted to be a part of all Fedora spins. It should not happen that a spin is missing a core application after a clean installation.
Workstation WG will want to have virtualization front-end, while KDE won't. Text-only spins won't want to a web browser. Etc.
* The presence check (that the apps are installed) could be easily done by OpenQA. Possibly, it could be done even for more DE than just Gnome, hence we could enhance the user experience for spin users.
Agreed.
* Functionality of the applications should be redefined - what we expect for them to be doing - and this functionality should be required. I suppose, that we would only block on Workstation functionality, though. However, these guidelines could help the spin teams to decide which apps to use as core apps.
I don't understand which apps and which functionality you talk about here. We already require basic functionality for all default desktop apps, and our criteria include required functionality in many tools/apps outside of the basic scope. What do you mean here?
* *Core applications* should not just have a basic functionality, as defined in *Gnome Menus testcase* (https://fedoraproject.org/wiki/QA:Testcase_desktop_menus), they should be fully functioning.
What does fully functioning mean? A slippery slope here :-) Either way, this needs to get developer backing, and I'm very skeptical about getting it.
Before I focus on details, I would like to know your opinions on this matter.
I'm missing one thing and perhaps I misunderstood some of this because of that. What is your main motivation here? What exactly are you trying to improve? Thanks.
_______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx
