Thank you all for your +1s. The change is live now. -- Mikolaj On Fri, Sep 13, 2024 at 10:07 PM Mikolaj Izdebski <mizdebsk@xxxxxxxxxx> wrote: > > Hi, > > Koschei has issues due to increased robots activity. In particular the > slow "affected-by" query seems to be problematic. > This causes user-visible downtime from time to time and I was unable > to fix it by simply scaling up the application. > More details: https://pagure.io/fedora-infrastructure/issue/12180 > > The issue causes significant service degradation to our users. > Therefore I think it's justified to add custom robots.txt for Koschei > before the freeze is lifted. > I would like to apply the following ansible.git patch: > > diff --git a/roles/httpd/website/templates/robots/koschei.fedoraproject.org-robots.txt > b/roles/httpd/website/templates/robots/koschei.fedoraproject.org-robots.txt > new file mode 100644 > index 0000000000..9a36280d39 > --- /dev/null > +++ b/roles/httpd/website/templates/robots/koschei.fedoraproject.org-robots.txt > @@ -0,0 +1,5 @@ > +User-agent: * > +Crawl-delay: 2 > +Disallow: /login > +Disallow: /bugreport > +Disallow: /affected-by > > And deploy the changes to proxies by running: > > # ansible-playbook /srv/web/infra/ansible/playbooks/groups/proxies.yml -t robots > > +1s to this FBR? > > -- > Mikolaj Izdebski -- _______________________________________________ infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
