+1 from me. Hopefully this permanently fixes the issue
On Wed, Sep 22, 2021 at 10:55 PM Kevin Fenzi <kevin@xxxxxxxxx> wrote:
On Wed, Sep 22, 2021 at 04:12:28PM -0400, Stephen John Smoogen wrote:
> On Wed, 22 Sept 2021 at 15:19, Kevin Fenzi <kevin@xxxxxxxxx> wrote:
> >
> > Yesterday we were having lots of issues with proxy01/10 in IAD2.
> > They would stop processing connections. Restarting httpd seemed to clear
> > it up for a while, then it would get stuck again.
> >
> > My current theory is that we were hitting the limit of 900 clients for
> > some reason and it wasn't processing them correctly when it got to that
> > point.
> >
> > So, I increased that limit to 1500 and also setup a SSL session cache
> > (which it was complaining about that we didn't have). Since then,
> > proxy01/10 with those changes have been running ok.
> >
> > I'd like to push this out to the other proxies now as well, as some of
> > them have been alerting from time to time and it could be this same
> > issue.
> >
> > I already pushed this commit because I wanted 01/10 to be in sync/in
> > git.
> >
> > +1's to push it to the rest of the proxies?
> >
>
> There is a second part to your change:
>
> > SSLStaplingCache shmcb:/tmp/stapling_cache(128000)
> > +SSLSessionCache shmcb:/run/httpd/sslcache(10240000)
>
> Is that part of this or something that got pulled in by accident?
Thats the "also setup a SSL session cache" part...
kevin
--
>
>
> > commit 313674646df60fc0e8342eff26094f694105cf76
> > Author: Kevin Fenzi <kevin@xxxxxxxxx>
> > Date: Tue Sep 21 16:19:14 2021 -0700
> >
> > proxies: increase max workers
> >
> > Also add a ssl connection cache.
> > These changes are live on proxy01/10 and seem to have made them stable
> > again. Will look at pushing to the rest tomorrow.
> >
> > Signed-off-by: Kevin Fenzi <kevin@xxxxxxxxx>
> >
> > diff --git a/inventory/group_vars/proxies b/inventory/group_vars/proxies
> > index c04531a57..5b0a25fee 100644
> > --- a/inventory/group_vars/proxies
> > +++ b/inventory/group_vars/proxies
> > @@ -7,7 +7,7 @@ num_cpus: 6
> > # This is used in the httpd.conf to determine the value for serverlimit and
> > # maxrequestworkers. On 8gb proxies, 900 seems fine. But on 4gb proxies, this
> > # should be lowered in the host vars for that proxy.
> > -maxrequestworkers: 900
> > +maxrequestworkers: 1500
> >
> > tcp_ports: [
> > # For apache, generally.
> > diff --git a/roles/httpd/proxy/templates/httpd.conf.j2 b/roles/httpd/proxy/templates/httpd.conf.j2
> > index 00947131f..5b1e0debf 100644
> > --- a/roles/httpd/proxy/templates/httpd.conf.j2
> > +++ b/roles/httpd/proxy/templates/httpd.conf.j2
> > @@ -773,3 +773,5 @@ EnableSendfile on
> >
> > # Configure a location for OCSP stapling
> > SSLStaplingCache shmcb:/tmp/stapling_cache(128000)
> > +SSLSessionCache shmcb:/run/httpd/sslcache(10240000)
> > +SSLSessionCacheTimeout 600
> >
> > kevin
> > _______________________________________________
> > infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
> > To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx
> > Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
> > Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
>
>
>
> --
> Stephen J Smoogen.
> I've seen things you people wouldn't believe. Flame wars in
> sci.astro.orion. I have seen SPAM filters overload because of Godwin's
> Law. All those moments will be lost in time... like posts on a BBS...
> time to shutdown -h now.
> _______________________________________________
> infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
_______________________________________________
infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
_______________________________________________ infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
