On Fri, Sep 13, 2024 at 10:07:40PM GMT, Mikolaj Izdebski wrote: > Hi, > > Koschei has issues due to increased robots activity. In particular the > slow "affected-by" query seems to be problematic. > This causes user-visible downtime from time to time and I was unable > to fix it by simply scaling up the application. > More details: https://pagure.io/fedora-infrastructure/issue/12180 > > The issue causes significant service degradation to our users. > Therefore I think it's justified to add custom robots.txt for Koschei > before the freeze is lifted. > I would like to apply the following ansible.git patch: > > diff --git a/roles/httpd/website/templates/robots/koschei.fedoraproject.org-robots.txt > b/roles/httpd/website/templates/robots/koschei.fedoraproject.org-robots.txt > new file mode 100644 > index 0000000000..9a36280d39 > --- /dev/null > +++ b/roles/httpd/website/templates/robots/koschei.fedoraproject.org-robots.txt > @@ -0,0 +1,5 @@ > +User-agent: * > +Crawl-delay: 2 > +Disallow: /login > +Disallow: /bugreport > +Disallow: /affected-by > > And deploy the changes to proxies by running: > > # ansible-playbook /srv/web/infra/ansible/playbooks/groups/proxies.yml -t robots > > +1s to this FBR? +1 here kevin
Attachment:
signature.asc
Description: PGP signature
-- _______________________________________________ infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
