Hi folks! So the F35 retrospective has been up for a while: https://fedoraproject.org/wiki/Fedora_35_QA_Retrospective I thought I'd send out my planned set of actions in response to the points raised there. Please suggest any improvements you can think of. I'll also put this on the agenda for the meeting on Monday, then I'll go ahead and implement these (with any improvements that are suggested) next week. 1. "adamwill - It took a long time to work through all the iterations of the fedora-third-party/SELinux bug. Seems like we could have tightened the loop on that somehow." - I don't think there's any specific action to take immediately here, but if a similar situation occurs in future I intend to be more hands-on about asking the relevant component developers and the SELinux developers to work faster to get to the bottom of the problem pile sooner. 2. "adamwill - Movement on the KDE blockers was initially slow, if we could have resolved them earlier we would have had more time to find and fix the bugs that eventually delayed Final." - I think we kinda addressed this during the cycle, by pulling in some folks who were not really involved in the blocker fix cycle previously. For F36 and later we now have more KDE folks we can tap to get problems solved promptly. 3. "Ahmedalmeleh - Bugzilla is challenging to me and still getting used to it. I wish I was able to learn how to operate OpenQA's automated tests, in the given timeframe and was given guidance sooner.", also Ahmed's 'wishlist' item and Matt's item - I'll plan to file a ticket for improved guidance on using Bugzilla, and ask Ahmed for specific input on what might help. I wonder if it might be a good idea to drag ourselves into the 21st century and record some videos covering common QA activities too, that might be something some folks would appreciate more than text instructions? For openQA, I'll ask Ahmed for more detail on specifically what he'd like to do with it here, as "operating" the tests isn't something we envisage most contributors typically needing to do. 4. "kparal - After F34, we talked in our team about doing a selected compose testing in a complete full run (all test cases), way ahead of the milestones (Beta and Final), and as a group effort to encourage people to really fill out all test cases. Yet we never got to it in F35. Some of the bugs might've been discovered earlier this way." - I wonder if we could do this as a "test day" or "test week", or two: schedule one event around Beta freeze and one event around Final freeze, and have the goal of the event be to fill out the test matrices. What do people think of this idea? If it seems like a good one, I'll file tickets for creating those events. 5. "Mattdm - The timing of KDE Plasma releases is not well-aligned with our schedule..." - this honestly doesn't seem like something QA can really take action on, maybe it's more suited to the wider retrospective Ben has been talking about? 6. "Mattdm - Lots of people (as non-scientifically measured by my personal impressions of help requests I'm seeing) are hitting the wireplumber-not-enabled problem. Maybe we should have blocked on that, actually." - I've already updated the commonbugs note on this, but another action we could take is to test upgrades from an older release and see if we can spot the "pulseaudio still enabled" case happening and if so what triggers it. Aside from that, I think we'll just have to keep this one on record as an example to keep in mind when making these kinds of subjective blocker calls in future. 7. "sumantrom - Sending some testing event updates to our "users" list. I see surge in contributors when there is an article posted in Fedora Magazine which means Fedora's user base wants to contribute/ try out things. Posting things in the user-list will help." - this seems like a good idea for sure. If we still work from an SOP for organizing these kinds of events, a concrete action would be to add this step to the SOP, I can file a ticket for that. 8. "sumantrom - Onboarding session on Bugzilla and test case writing." - again this seems like a good idea, and we can file a ticket to have such an event. Perhaps we could also link this up with point 3) and record the session to use, perhaps after editing it down, as a "getting started with Bugzilla" video? 9. "frantisekz - Some testing could have been conducted earlier in the development, especially KDE/Discover blockers and bugs have been present for a long time before the release." - I think the practical step we can take here is already in progress: add more specific package manager criteria and test cases. See the "new criterion proposal: Graphical package managers" thread. Once the new criteria are decided, we can create new test cases or extend existing ones, and once that's done, the testing should get done more often and earlier. We can even automate it. I can file tickets for those actions. How does this sound to everyone? Please add any suggestions you have! And there's still time to add new items to the retrospective itself, if any new ones show up, I'll update this list of proposed actions. Thanks everyone! -- Adam Williamson Fedora QA IRC: adamw | Twitter: adamw_ha https://www.happyassassin.net _______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
