Good Morning Everyone, Kevin and I had a discussion on IRC the other day about mirroring the ansible repo. The options we considered are: - mirroring to pagure - mirroring from pagure - using pagure's mirroring feature - using a different approach - Mirroring to pagure This meant, the canonical place for ansible remains in the batcave and pagure only periodically pulls from there to update itself (or a hook updates pagure on push). This would work, but it basically make the pull-request workflow for reviewers a little more complex, as you would have to download the patch of the PR, apply it (potentially adding the: Merging <id> so it marks the PR as merged) and then push to the batcave. If you merged via the UI in pagure, you would then have to pull from pagure and then push to the batcave, with the risk of a race condition if someone or something updates pagure right after your merge and suddenly the commit(s) of the PR disapear (since the content in pagure would be overriden by what is coming from the batcave). So, not an ideal workflow. - Mirroring from pagure This means, the canonical place for ansible is in pagure.io. However, we need to still be able to run ansible when pagure.io goes down (keeping in mind batcave and pagure.io are in two different data-center). So we want to keep a mirror of the ansible repo in the batcave for emergencies and that repo should be as up to date as possible. With this approach, all the work happens on pagure.io, the PR workflow is the usual, straight forward one and we have a mirror in the batcave that is mostly up to date. - Using pagure's mirroring feature Pagure can mirror in and out, out is done right after a push, however, batcave is not accessible from pagure.io, so we just can't use this. - Using a different approach We have two ways we could do this: a simple cron job, a message-based service. For a first step I went with a third approach: a small python service that runs every 3 minutes (configurable): git fetch && git fsck (to ensure the git is in a correct state). It actually doesn't run every 3 minutes, it calls git, then wait for 3 minutes then calls git again and so on. So if git was ever to takes 4 minutes to run, there is no risk of the program stepping on its own toes. I would like to propose that install and configure this for a test. I propose that we keep /git/ansible.git as is and just have a /git/mirrors/ansible.git which will be the mirror from pagure. We can migrate the hook from /git/ansible.git to /git/mirrors/ansible.git so /srv/web/infra/ansible (ie: the exploded tree) is coming from the mirror. I guess the lag time to get /git/mirrors updated will quickly be annoying (up to 3 minutes between when the PR is merged and when the playbook is updated), so if we like the workflow we will likely want to switch pretty quickly to a message-based service and then we could keep the current cron-like service to run hourly and update /git/ansible.git which would then become some kind of backup. What do you think? FBR worthy? (ie: if F32 goes out next week, I think we can wait, if it doesn't do we want to try this sooner?) If we like the idea, I'll start looking into the fedora-messaging based consumer, using the approach from the current service, it should be pretty straight forward (I'll re-use the same project for it). Oh, and if you want to see the code of the current service: https://pagure.io/Fedora-Infra/mirror_from_pagure Thanks, Pierre _______________________________________________ infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
