On Fri, 17 Apr 2020 at 14:22, Leonardo Rossetti <lrossett@xxxxxxxxxx> wrote:
On Fri, Apr 17, 2020 at 8:07 AM Clement Verna <cverna@xxxxxxxxxxxxxxxxx> wrote:Hi all,I wanted to start a discussion and possibly some work around automating the manual tasks involved in the release engineering work.In particular I have in mind the following tasks :- processing the unretire package tickets.- processing the requests for a new package or a new branch.- container base image release.Instead of looking at each of these individually I was thinking that it might be interesting to look at having an automation framework or something like a bot that could be flexible enough to add more tasks or process in the future.To do that we have different possibilities, one could be to build a bot that has a similar architecture than the compose-tracker (https://pagure.io/releng/compose-tracker) ie a fedora-messaging consumer processing messages.Another option is to use loopabull (https://github.com/maxamillion/loopabull) to trigger ansible playbook on fedora-messaging messages.Both solutions are quite similar, but one (loopabull) provides already the boilerplate to trigger a script or a function based on messages received (a bit like AWS lambda or other serverless framework). We also have already a few process automated that way (https://pagure.io/Fedora-Infra/loopabull-tasks).So I believe that using loopabull might be the best way forward, but I would be interested to hear about other ideas :-)I would lean to use loopabull because it already works in a "reactive way" with mq messages and we don't need to write a full application since we will be using ansible (which still can be "extended" developing modules for complex tasks) - the above script could become a couple of ansible modules to be used in a playbook with loopabull.Now if we look at the tasks to automate, I was thinking that we could implement that automation in different phases :un-retiring tickets:
- First step would be to run automatically the check-unretirement script (https://pagure.io/releng/blob/master/f/scripts/check-unretirement.py) and redirect its output into the ticket comments. That way people processing the ticket have all the information available in the ticket.
- Second step would be to process automatically the tickets that do not require a new bz review (retired less than 8 weeks ago)
- Finally see if we can process automatically all the tickets.
creating new dist-git repo or branches:
- Idea here would be to run the fedscm-admin (https://pagure.io/fedscm-admin) cli automatically when a new ticket is created here (https://pagure.io/releng/fedora-scm-requests).
I think it would also need to check for missed tickets from time to time, like every 5 minutes for example, in case it missed one for whatever reason.container base image release:
- Instead of building the image every night, we could rebuild the image only when at least 1 package present in the base image has been updated.
- Push the image weekly to the registry if a build happened during that week.
Is there any particular reason for using a weekly push schedule (assuming doing one on every build is way too much)?
So yeah that comes down to how update works for the Docker Hub official images, this is done by PR and need someone from Docker to review and approve the PR, so we can't really spam them with multiple PR per week. See https://github.com/docker-library/official-images/issues/7529 for more info.
Also I think there is little point to push a new image every time there is at least one package updated, doing it weekly allows to release the latest image that might contain multiple updates. Longer term we could be smarter and have a special case for security update.
_______________________________________________Again here are some ideas, and I would very much appreciate feedback or other ideas :-). Also if you think about another process that could be automated that way please share it :-).ThanksClément
rel-eng mailing list -- rel-eng@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to rel-eng-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/rel-eng@xxxxxxxxxxxxxxxxxxxxxxx
--
_______________________________________________ infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
