My initial opinion is that it's not a useful idea, for two reasons:
1) Even with this split, automating any of this with openQA is *not*
particularly easy, unless I'm missing something. App icons change, and
the set of apps included in the live images changes. If you just make a
test which literally has needles for every single app icon and just
goes through a loop of opening the 'start menu', looking for an icon,
clicking on it, closing the resulting app, and going onto the next one,
that test is going to break *any time* an app icon changes or an app is
removed from the image. (There's also the possibility that background
or transparency changes break *ALL* the icon needles at once, which
would be a nightmare).
What if we started the applications with Alt+F2? That works and does not need to check
for icons.
2) How do you know for sure that an app has launched? And how do you
quit out of it? They *mostly* cause a window with standard chrome to
appear...but not all of them do. They *mostly* launch to some state
where you can just click an 'X' button to quit...but not all of them
do. The exceptions would be an additional bunch of fragile complexity
in the test.
I'd think to use Alt+F4 - no need for 'X' buttons.
3) Even if we could deal with 1) and 2) somehow, having just one part
of the test automated isn't a huge win. Just knowing that the apps
launch and close isn't *that* useful...and having that part of the test
automated doesn't make it any *easier* for a human to do 3), really.
So, where's the win?
I agree that this does not solve the entire problem, therefore I have also proposed to focus on a given set of applications that we would test thoroughly and with the rest just do some really basic stuff. When you take a look at the statistics, the menu test is one of the least frequently tested ones. And we know why - nobody wants to click through all of this stuff, because half of the default gnome applications nobody ever really uses.
If the test case should do its job, we need to test it often and start early in order not to catch bugs just after the Final freeze. Otherwise, it has no sense anyway. Maybe we can find some other solution. Anyway, this proposal was mainly to start a discussion, because if anything should change, we better start solving it right now, otherwise we will end up with nothing for Fedora 30. But, if we cannot find anything better ... then we still can go the usual way.
I'm willing to be convinced, though. That's just my initial thinking
(and why I always dismissed this line of work when I thought about it
before).
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
_______________________________________________
test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx
--
Lukáš Růžička
FEDORA QE, RHCE
Purkyňova 115
612 45 Brno - Královo Pole
_______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx
