On Sun, Nov 27, 2022 at 5:32 PM Kevin Fenzi <kevin@xxxxxxxxx> wrote: > > C) Same as B, but with an external builder > > We already build the new websites on Gitlab CI, and since the S3 gateway is accessible from the outside, we could just push the build artifacts to s3 directly from GitLab CI. Then sync the proxies from it. > > Or just pull from gitlab directly? Or does it expose that data in a way > we could sync from? Gitlab CI supports downloadable artifacts. A tarball seems most appropriate here but I often build RPMs myself. :-D Here's a simple CI script that build and tars the website up: https://gitlab.com/tchollingsworth/fedora-websites/-/blob/6787d86f6e81c849dfaafd245878bd16710e9bb4/.gitlab-ci.yml And a stable link by branch and commit to the tarball it generated: https://gitlab.com/api/v4/projects/tchollingsworth%2Ffedora-websites/jobs/artifacts/master/raw/sites/getfedora.org/getfedora.org.tar.gz?job=build https://gitlab.com/api/v4/projects/tchollingsworth%2Ffedora-websites/jobs/artifacts/6787d86f6e81c849dfaafd245878bd16710e9bb4/raw/sites/getfedora.org/getfedora.org.tar.gz?job=build > > We could also improve B and C by adding fedora-messaging to the mix to trigger a proxy resync as soon as a new build is available instead of doing so every hour. > > +1 (although we should make sure not to thundering herd the endpoint if > all proxies decided to sync at the same instant). Gitlab can call a webhook when the build is completed, which could trigger (a bus message that fires off) a sync. I see CentOS developed webhooks that forward commits, issues, etc. to the fedora-messaging bus, not sure if they were ever put into production or if they include CI successes and failures, but maybe half the work needed for this is already done. > Yeah, although it works the other way too... if we move to gitlab it > would need someone who understands that setup to debug and fix. I spent more time messing around with stuff like rolling back to Fedora 34 because the flask-assets dependency it needed was retired in the current release than I did with anything that was gitlab's fault. ;-) On Mon, Nov 28, 2022 at 6:24 AM darknao <darknao@xxxxxxxxxxxxxxxxx> wrote: > One downside to that is every build creates a new image (or at least a > new layer), that proxies will have to pull every time. > With rsync (or s3 sync) we only download what actually changes and save > bandwidth in the long run. Maybe my build isn't working all the way but tarball above is 11MB gzipped and 30MB uncompressed so probably not a big burden, especially if you only build when someone pushes. _______________________________________________ infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
