On Wed, 2018-11-07 at 10:42 +0100, Lukas Ruzicka wrote: > Hello Fedora QA and friends, > > the *Desktop Menus Testcase* is one of the most boring tests to do and it > would be nice if OpenQA could do it for us. However, how the testcase is > written, it cannot be automated because it relies on a human mind too much. > I am proposing to split it into the following testcases: > > 1. *Desktop Menus Visuals* in which we would check if all menu entries have > proper icons, names, etc. > 2. *Desktop Menus AppStarts* in which we would test if all menu items can > be started (easy with OpenQA) > 3. *Desktop Menus Basic Functionality* in which we would test if the apps > basically do work (perhaps can be automated in OpenQA if the basic > functions are not too complicated) > > Please, let me know what you think about it. My initial opinion is that it's not a useful idea, for two reasons: 1) Even with this split, automating any of this with openQA is *not* particularly easy, unless I'm missing something. App icons change, and the set of apps included in the live images changes. If you just make a test which literally has needles for every single app icon and just goes through a loop of opening the 'start menu', looking for an icon, clicking on it, closing the resulting app, and going onto the next one, that test is going to break *any time* an app icon changes or an app is removed from the image. (There's also the possibility that background or transparency changes break *ALL* the icon needles at once, which would be a nightmare). 2) How do you know for sure that an app has launched? And how do you quit out of it? They *mostly* cause a window with standard chrome to appear...but not all of them do. They *mostly* launch to some state where you can just click an 'X' button to quit...but not all of them do. The exceptions would be an additional bunch of fragile complexity in the test. 3) Even if we could deal with 1) and 2) somehow, having just one part of the test automated isn't a huge win. Just knowing that the apps launch and close isn't *that* useful...and having that part of the test automated doesn't make it any *easier* for a human to do 3), really. So, where's the win? I'm willing to be convinced, though. That's just my initial thinking (and why I always dismissed this line of work when I thought about it before). -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx
