On Wed, May 4, 2022 at 2:15 AM Adam Williamson <adamwill@xxxxxxxxxxxxxxxxx> wrote:
> 1. I know we have had GNOME Test days, but this
> stuff didn't come up. Presumably, it would have if someone had happened
> to look at GNOME Photos. Can we formally go through
> https://www.fedoraproject.org/wiki/QA:Testcase_desktop_app_basic and
> https://www.fedoraproject.org/wiki/QA:Testcase_desktop_app_basic_others
> much, much earlier (probably when the GNOME pre-release is available),
> either as part of a test day or some other formal thing? Or is there
> something else going on here that I'm not looking closely enough to see?
Well, kinda, yes. The thing going on is that we *did* go through that
test at several earlier points:
https://openqa.fedoraproject.org/testcase_stats/36/Desktop/QA_Testcase_desktop_app_basic_others_Release_blocking_desktops___lt_b_gt_x86___x86_64_lt__b_gt_.html
but these bugs weren't discovered at that time. This is likely because
of your idea 2: "basic functionality" is a bit up for debate. You can
take an extremely minimalist approach to this (run the app, click a
couple of buttons, say it's OK if nothing explodes and no babies are
eaten) or a slightly more maximalist one (run the app, and actually try
and do something useful with it). In this case, before Final, when we
ran this test we mostly did the minimalist thing. At Final RC stage, we
went a bit more maximalist.
I wouldn't call that maximalist, that would mean testing everything possible. A *realistic* approach is more appropriate, I think. To "do something useful with it" is a great description. What good is it if it can start and survive a few button clicks, if you can't do useful tasks? The tasks it was actually designed for. What's the point of shipping a photo organizer where you can't create and organize albums? This is the realistic scenario, and ideally the thing we would always try to test. Of course, not always people have time to do that, and only some pathways can be broken while not others.
Discovering bugs in the realistic scenarios requires time and also some luck. At the same time, bugs in realistic scenarios will very likely prevent actual users from actually using that app, even if just certain pathways are broken. Even if some of those bugs seem trivial to discover, they are not. If you edit any other field than the email address in gnome-contacts, you'll not see contacts duplication. So as a QA, you only have a certain chance to spot it. In the real world with regular usage, however, sooner or later you'll definitely edit someone's email address and then you'll see that contact doubled. (This is just a nice example, this particular bug was not accepted as a blocker, as it is not ground-breaking).
_______________________________________________ desktop mailing list -- desktop@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to desktop-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/desktop@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
