We've talked about replacing blocker bug review meetings with something else for a long time. The meeting has an upside of a higher communication bandwidth, but also a downside of requiring participants to be available at the same time (throughout the world), often being extremely long, and being scheduled just once per week (because of mentioned downsides). I've put some thoughts into making the process asynchronous, and you can see all the options I considered described below. I close up with a proposal for the best choice I found.
Asynchronous blocker review process will:
- allow more people to participate (they are not restricted to a particular time, which is inconvenient for some people)
- allow us to better gather feedback from maintainers/developers by CCing them
- possibly allow a faster turn-around for voting (if you propose a blocker on Tuesday, it doesn't have to wait for the next Monday's meeting)
- reduce frustration from long meetings (but perhaps add different ones)
- restrict our communication bandwidth somewhat (real-time chat makes it easier to debug issues than e.g. email)
I think we'll need to try it first to decide if we like it or not. Here are my thoughts for different systems we could use:
Blocker Bugs App (DIY solution)
******************
We can implement all the necessary features to our Blocker Bugs App [1]. We even talked about it in the past. There would be a Voting page where discussions could be held for each proposed blocker, and a voting system. We could tailor everything exactly to our needs. But it also implies a lot of time spent in development, writing from scratch features that already exist elsewhere (discussions, notifications, etc) and a long-term maintenance of it (we want out blocker decisions to be available basically indefinitely, ideally with a URL that doesn't change). I'm skeptical that we want to devote a lot of our time to implement such a system, and I don't even think it's a smart thing to do, if there are existing solutions (implemented and maintained by other people) which could work "well enough". That's why I'm mentioning this solution, but try to avoid it as much as possible, and explore other systems below.
[1] https://qa.fedoraproject.org/blockerbugs/
Mailing lists
************
This is trivial to use. For every proposed blocker, we simply start a thread on test list (with a predefined subject format), discuss & decide. It's simple, but also quite bare-bones. We can CC people easily, but that tends to be complicated with people not subscribed to the list. It's very easy to see just a partial conversation, due to the usual reply headers madness. We can link this conversation into the bug report using a hyperkitty view (as long as we have hyperkitty), but it suffers from the same issues, possibly showing just a partial conversation. It's very hard to count the votes across all the replies. It's also hard to track the status of the conversation (initial discussion, voting in progress, resolved, etc). Overall, mailing lists are workable, but far from a good solution, and that's why I'm looking into other solutions below.
Bugzilla
********
For each proposed blocker, we can create (ideally auto-create) a parallel bug report in Bugzilla (against "distribution" or a special component that we create). We will link those two bugs together, using Blocks or External Trackers. Then we simply have the conversation there. We can teach Blocker Bugs App to link to the "discussion bug" from its UI. We can easily CC and "needinfo" other people. Bugzilla automatically sends notifications for new comments, and anyone can subscribe to all blocker bug discussions (either by using Bugzilla's own component watcher, or by watching the appropriate distgit component in Pagure, if available). You can rely on having long-term stable URLs. You can list all open and closed blocker discussions tickets. We can use some additional status for marking "voting in progress" (though that won't be fully obvious). This all sounds pretty good. A slight drawback is that it's not easy to count votes - you have to go through all the comments, find the votes and manually count them. We could create a script that would parse all comments and update the vote summary in the Whiteboard (or perhaps Environment or Doc Text, which is multiline) after each discussion update. I believe Bugzilla is a decent solution for async blocker bug meetings. Its UI is not very pretty, but it has most of the necessary functions.
Pagure
*******
This is very similar to the Bugzilla description. For each proposed blocker, we (auto-)create a ticket in a "fedora-blockers" project, for the purpose of a blocker discussion. We interlink the bug and the ticket. We teach Blocker Bugs App to link to the discussion ticket from its UI. We can easily ping other people using @name tagging. Pagure automatically sends notifications for new comments, and anyone can subscribe to all blocker bug discussions using Pagure UI of the project. We can rely on having a long-term stable URLs. You can easily list all open and closed blocker discussion tickets. We can use tags to nicely distinguish tickets, for example blockers vs freeze exceptions, or the state of the conversation (initial discussion, voting in progress, accepted, rejected, etc). The modern UI with tags is very helpful here, allowing us to have a modern and an intuitive process. We still can't count votes easily, but we can create a script that will parse all comments and update the vote summary in the initial description after each ticket update. Thanks to FAS integration, selected people (e.g. members of a FAS group) can have elevated privileges in these tickets allowing them to control the blocker watcher script, similarly to how we control the IRC bot during IRC meetings. I see Pagure as a similar option to Bugzilla, just with much better UI/navigation and a few extra shinies.
The imagined workflow could look like this:
1. A blocker bug is proposed (using either Bugzilla directly or our Blocker Bugs App).
2. Blocker Bugs App (BBA) detects the new blocker and creates a new ticket in Pagure in the "fedora-blockers" project, then updates the bug to link to this ticket (a new comment, a Links entry). It shows both the bug and the discussion ticket in its UI.
3. All fedora-blockers subscribers are notified of the new ticket (i.e. the newly proposed blocker).
4. People discuss the problem in the Pagure ticket.
5. Once some kind of understanding of the issue is formed, a privileged member (e.g. a member of @fedora-qa FAS group) can start the vote by including a command in his comment, e.g. "START VOTE" on a separate line. (Note that this is just a proposal. We can easily let anyone start the vote, or we can allow voting right from the beginning without an explicit start.)
6. The blocker bot (probably part of BBA) listens to all fedora-blockers Pagure tickets changes, and once it detects a vote start command, it updates the ticket e.g. with a "vote in progress" tag.
7. People vote by submitting comments containing VOTE +1/0/-1 on a separate line (and including any justification or feedback they wish in the comment as well; the command has to simply be on its own line so that we can detect it well).
8. The blocker bot listens to ticket updates and after each vote command, it updates the ticket's initial description with a summary of the vote (how many people voted for and against, including names, and hyperlinks to their comments). People can easily correct their vote by sending another comment, the bot will use the last one. People can easily see the vote status by just looking at the ticket's initial description.
9. If new important information appear or the vote needs to be repeated for any other reason, a privileged member can restart the vote e.g. by issuing RESTART VOTE command. (We can have more such administrative commands for any purpose we need, same as we have them with IRC bots).
10. Once the vote has been held for a long enough time (we can either set some standards, or it can be up to QA members judgment), a privileged member can end the vote by issuing AGREED <RESOLUTION> command. The <RESOLUTION> part will be standardized, it can look like "AGREED RejectedBlocker AcceptedFreezeException Some explanation here".
11. The blocker bot will detect this command and update the bug in Bugzilla by adjusting necessary fields, adding a comment that includes the provided explanation and linking to the Pagure ticket once again. It will also update the Pagure ticket, i.e. update necessary tags (e.g. "RejectedBlocker", "AcceptedFreezeException") and close the ticket.
12. goto 1
Note that we don't need the blocker bot to be fully or even partially implemented, just to try this out. We can easily do the bot steps manually in the beginning and decide whether we like this process, before we start working on the automation. We can pretty much start testing this right now, if we want.
What are your thoughts? Does the proposal sound reasonable? Would you change something? Would you use a different backend system? Have I forgotten something important? Feedback welcome.
Thanks,
Kamil
Asynchronous blocker review process will:
- allow more people to participate (they are not restricted to a particular time, which is inconvenient for some people)
- allow us to better gather feedback from maintainers/developers by CCing them
- possibly allow a faster turn-around for voting (if you propose a blocker on Tuesday, it doesn't have to wait for the next Monday's meeting)
- reduce frustration from long meetings (but perhaps add different ones)
- restrict our communication bandwidth somewhat (real-time chat makes it easier to debug issues than e.g. email)
I think we'll need to try it first to decide if we like it or not. Here are my thoughts for different systems we could use:
Blocker Bugs App (DIY solution)
******************
We can implement all the necessary features to our Blocker Bugs App [1]. We even talked about it in the past. There would be a Voting page where discussions could be held for each proposed blocker, and a voting system. We could tailor everything exactly to our needs. But it also implies a lot of time spent in development, writing from scratch features that already exist elsewhere (discussions, notifications, etc) and a long-term maintenance of it (we want out blocker decisions to be available basically indefinitely, ideally with a URL that doesn't change). I'm skeptical that we want to devote a lot of our time to implement such a system, and I don't even think it's a smart thing to do, if there are existing solutions (implemented and maintained by other people) which could work "well enough". That's why I'm mentioning this solution, but try to avoid it as much as possible, and explore other systems below.
[1] https://qa.fedoraproject.org/blockerbugs/
Mailing lists
************
This is trivial to use. For every proposed blocker, we simply start a thread on test list (with a predefined subject format), discuss & decide. It's simple, but also quite bare-bones. We can CC people easily, but that tends to be complicated with people not subscribed to the list. It's very easy to see just a partial conversation, due to the usual reply headers madness. We can link this conversation into the bug report using a hyperkitty view (as long as we have hyperkitty), but it suffers from the same issues, possibly showing just a partial conversation. It's very hard to count the votes across all the replies. It's also hard to track the status of the conversation (initial discussion, voting in progress, resolved, etc). Overall, mailing lists are workable, but far from a good solution, and that's why I'm looking into other solutions below.
Bugzilla
********
For each proposed blocker, we can create (ideally auto-create) a parallel bug report in Bugzilla (against "distribution" or a special component that we create). We will link those two bugs together, using Blocks or External Trackers. Then we simply have the conversation there. We can teach Blocker Bugs App to link to the "discussion bug" from its UI. We can easily CC and "needinfo" other people. Bugzilla automatically sends notifications for new comments, and anyone can subscribe to all blocker bug discussions (either by using Bugzilla's own component watcher, or by watching the appropriate distgit component in Pagure, if available). You can rely on having long-term stable URLs. You can list all open and closed blocker discussions tickets. We can use some additional status for marking "voting in progress" (though that won't be fully obvious). This all sounds pretty good. A slight drawback is that it's not easy to count votes - you have to go through all the comments, find the votes and manually count them. We could create a script that would parse all comments and update the vote summary in the Whiteboard (or perhaps Environment or Doc Text, which is multiline) after each discussion update. I believe Bugzilla is a decent solution for async blocker bug meetings. Its UI is not very pretty, but it has most of the necessary functions.
Pagure
*******
This is very similar to the Bugzilla description. For each proposed blocker, we (auto-)create a ticket in a "fedora-blockers" project, for the purpose of a blocker discussion. We interlink the bug and the ticket. We teach Blocker Bugs App to link to the discussion ticket from its UI. We can easily ping other people using @name tagging. Pagure automatically sends notifications for new comments, and anyone can subscribe to all blocker bug discussions using Pagure UI of the project. We can rely on having a long-term stable URLs. You can easily list all open and closed blocker discussion tickets. We can use tags to nicely distinguish tickets, for example blockers vs freeze exceptions, or the state of the conversation (initial discussion, voting in progress, accepted, rejected, etc). The modern UI with tags is very helpful here, allowing us to have a modern and an intuitive process. We still can't count votes easily, but we can create a script that will parse all comments and update the vote summary in the initial description after each ticket update. Thanks to FAS integration, selected people (e.g. members of a FAS group) can have elevated privileges in these tickets allowing them to control the blocker watcher script, similarly to how we control the IRC bot during IRC meetings. I see Pagure as a similar option to Bugzilla, just with much better UI/navigation and a few extra shinies.
The imagined workflow could look like this:
1. A blocker bug is proposed (using either Bugzilla directly or our Blocker Bugs App).
2. Blocker Bugs App (BBA) detects the new blocker and creates a new ticket in Pagure in the "fedora-blockers" project, then updates the bug to link to this ticket (a new comment, a Links entry). It shows both the bug and the discussion ticket in its UI.
3. All fedora-blockers subscribers are notified of the new ticket (i.e. the newly proposed blocker).
4. People discuss the problem in the Pagure ticket.
5. Once some kind of understanding of the issue is formed, a privileged member (e.g. a member of @fedora-qa FAS group) can start the vote by including a command in his comment, e.g. "START VOTE" on a separate line. (Note that this is just a proposal. We can easily let anyone start the vote, or we can allow voting right from the beginning without an explicit start.)
6. The blocker bot (probably part of BBA) listens to all fedora-blockers Pagure tickets changes, and once it detects a vote start command, it updates the ticket e.g. with a "vote in progress" tag.
7. People vote by submitting comments containing VOTE +1/0/-1 on a separate line (and including any justification or feedback they wish in the comment as well; the command has to simply be on its own line so that we can detect it well).
8. The blocker bot listens to ticket updates and after each vote command, it updates the ticket's initial description with a summary of the vote (how many people voted for and against, including names, and hyperlinks to their comments). People can easily correct their vote by sending another comment, the bot will use the last one. People can easily see the vote status by just looking at the ticket's initial description.
9. If new important information appear or the vote needs to be repeated for any other reason, a privileged member can restart the vote e.g. by issuing RESTART VOTE command. (We can have more such administrative commands for any purpose we need, same as we have them with IRC bots).
10. Once the vote has been held for a long enough time (we can either set some standards, or it can be up to QA members judgment), a privileged member can end the vote by issuing AGREED <RESOLUTION> command. The <RESOLUTION> part will be standardized, it can look like "AGREED RejectedBlocker AcceptedFreezeException Some explanation here".
11. The blocker bot will detect this command and update the bug in Bugzilla by adjusting necessary fields, adding a comment that includes the provided explanation and linking to the Pagure ticket once again. It will also update the Pagure ticket, i.e. update necessary tags (e.g. "RejectedBlocker", "AcceptedFreezeException") and close the ticket.
12. goto 1
Note that we don't need the blocker bot to be fully or even partially implemented, just to try this out. We can easily do the bot steps manually in the beginning and decide whether we like this process, before we start working on the automation. We can pretty much start testing this right now, if we want.
What are your thoughts? Does the proposal sound reasonable? Would you change something? Would you use a different backend system? Have I forgotten something important? Feedback welcome.
Thanks,
Kamil
_______________________________________________ test mailing list -- test@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to test-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@xxxxxxxxxxxxxxxxxxxxxxx
