-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 All, I've been using mod_authnz_ldap for a while with OpenLDAP over TLS and things have been going fairly well. This weekend, however, I had 3 servers stop working (returning HTTP 500 responses) for pages which were protected with HTTP Basic auth + mod_authnz_ldap. Two of them were unavailable while the third one was working, and then the third one stopped working as well. I checked everything: connectivity was okay (checked with openssl), certs were okay (also checked with openssl), and I even installed OpenLDAP-clients on the server to check to see if 'ldapsearch' worked (it did). After several "graceful" restarts, nothing was working. On a particularly low-activity server, I ran "/etc/init.d/httpd restart" and the page came right back up with no errors. Here's what I think has happened. First, we are using Let's Encrypt for not only our httpd server certificates but also for the OpenLDAP server. Second, our most recent certificate for the LDAP server has a "not before" date of "Oct 28 09:47:18 2018 GMT". That's very likely to be roughly the time all of this started becoming a problem. Note: We are using "LDAPVerifyServerCert Off" though that it a vestige of having self-signed certificates before moving to Let's Encrypt. So the certificate chain itself shouldn't be a problem. Third, the two servers that stopped working (roughly) at the same time have a system clock tracking UTC. Before restarting httpd on the second of the two servers, I grabbed the date-of-last restart -- that is, the timestamp of the oldest httpd process on the box: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 15973 0.1 0.6 415204 24060 ? Ss Oct22 12:29 /usr/sbin/http So only graceful restarts have occurred since Oct 22... before the certificate was re-issued. Fourth, we use logrotate and the latest logrotation on the initially-broken servers (using log-file timestamps) shows Oct 28 03:51 UTC. For the server that *wasn't* suffering until later, we have: tz: America/New_York (/etc/timezone... /etc/localtime says UTC) logrotate @ Oct 28 03:33 UCT logrotate performs "/sbin/service httpd reload" which I believe performs a "graceful" reload of httpd. These things don't entirely line-up... I was thinking I'd find that logrotate on one server ran on a several-hour delay from the other two and that would explain why one server was working while the others weren't, but it appears that logrotate ran roughly at the same time on all of the servers (at least, within a 30 minutes or so). Is anyone aware of any issues with mod_authnz_ldap caching TLS certificates or anything like that? I literally changed nothing on the LDAP server side or the httpd server side, but an httpd "restart" fixed this problem while no number of "graceful" reloads seemed to improve things. Thanks, - -chris -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlvXC3wACgkQHPApP6U8 pFh/cRAAq10ZkYWnI5Rzjrmx93DCRxRajo/XXmFC/9LeJEylCtLslmUd7oBqTSBn ikV56lWRIoGjVUVG3rGRConQzV76gbokai5q39a3Pf4mcQ2a20pec34aHtNNf7gE j4YFmSqWnAhZ6NHVAQZg4zLxV4JCS/OrVbU9jlgevZhXhaOFI5cwiCz51HyPE1z6 OPqeXz91P4AJvIH7hd5tXHmIVRopjOhJID15gGkzlNIRNrB00OMGFcTQbcUhzLj+ jClAdsbMCYnjuKRvTUnNVwTTBujj9G6wPBgvt0egdk15UhUDRyJnVciNYF85VgeX 66WrMB0GiPDepNTABWSj1gDd5pNaXd7gun2Lhbn3X7oAvawlYAAvsglvXbJedIzA y4gJgxn2zd9PU3F2Wf58j/zmMv7YQijcPT4yI+yWTwfYNx+QlcCVboxusY/i8YNE iN8lUFqh6kwICJazyLXpxNW6gMNoEEW9BRpaEC2doF1pxNk6LmI8ZdN5d24TMyj3 xTMje7jiNA7moYxx2wUR081IpyJezHzTvtmrKfnWz1szI5uYadott0eRq+i6JGw+ 7fb3OmA++MJUeB+uM5/O1QndR5zQ8B9XJ4cVTqaZLdG2CIiPJp4T6+D7qWpyWnEi Qbb0WxFauNFgsN417qrnEjjXLFAdeFJwq81sv4UTtaP3Yfhla74= =MvMp -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
|