Is it really that easy? I see the following issues:* You can implement a test for each application that matches some portion of its interface to make sure the app started successfully. E.g. for Gedit that could be Open button in top left, or for Nautilus that could be a portion of its side bar. That will require frequent maintenance, though (every time the font changes, or widgets move a few pixels, etc).
True. However, SUSE do that on all their graphical applications, so I believe there must be a way to do it properly. For example, SUSE use the About menu of the application to check and they use Keyboard. This might be a way to go.
This is the OpenSUSE code to test it:
sub run {
assert_gui_app('brasero', install => !is_sle, remain => 1);
# check about window
send_key 'alt-h';
wait_still_screen 3;
send_key 'a';
assert_screen 'brasero-help-about';
send_key 'alt-f4';
wait_still_screen 3;
send_key 'alt-f4';
}
assert_gui_app('brasero', install => !is_sle, remain => 1);
# check about window
send_key 'alt-h';
wait_still_screen 3;
send_key 'a';
assert_screen 'brasero-help-about';
send_key 'alt-f4';
wait_still_screen 3;
send_key 'alt-f4';
}
In Gnome, I realized that I could use Super+F10 to get the menu with the About item, click on it and then I could look for the program icon which would also be able to test that an icon is there.
Furthermore, I can't imagine how you'd make sure that *all* applications have been really tested, using OpenQA. Let's say there are 20 default applications in F29 Workstation, and you implement the test for all of them. If a new default application is added in F30 (so there are 21 of them), OpenQA can't spot that - it will claim that all applications start successfully, but it only covers all apps known to OpenQA, not all apps available in that environment. So this will need regular verification, which is unfortunately quite easy to forget about.
Well, yes. It will. However, the situation, as it is now, needs regular testing, which is likely to be put aside until the very last day before freeze (and I am sure the test statistics do prove it - https://www.happyassassin.net/testcase_stats/28/Desktop/QA_Testcase_desktop_menus_Release_blocking_desktops___lt_b_gt_x86___x86_64_lt__b_gt_.html), for this is the most monkeylike testing business we have and it really takes strong nerves to do it. With automation, we still would have to keep an eye on that, but we could be doing that once per release and still be able to test the majority of daily composes.
3. *Desktop Menus Basic Functionality* in which we would test if the apps basically do work (perhaps can be automated in OpenQA if the basic functions are not too complicated)As long as we do this manually, I'm not sure what this proposal really brings us, except for (ironically) more bureaucracy work with the matrices. If I test basic functionality manually, I automatically see whether the menu item visuals are OK (name, icon) and whether the app can be started (kinda prerequisite for basic functionality:)). So I'm not sure where we save work with separated test cases. Can you clarify where you see the time saved with this?
Yeah, you are right here, again. So why not take it a step further:
We could create a submatrix of all basic applications (that does not change with a compose) and write a simple terminal script that would be able to report the installed version, the date, and the result of the test case to that matrix, so anybody who would work with one of those applications could run:
desktoptest fail gnome-maps, or
desktoptest pass gnome-maps
and all results would be collected, so after some time, we could see how many and when were tested and for the compose matrix we could be testing just those which have not been tested otherwise.
--
Lukáš Růžička
FEDORA QE, RHCE
Purkyňova 115
612 45 Brno - Královo Pole
_______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx
