I'm having a perplexing failure authenticating users against LDAP on one server. I have a second server that has the same Apache configuration, authenticates against the same LDAP server with the same bind account, and succeeds. [Fri Aug 05 17:46:23 2011] [info] Server: Apache/2.2.3, Interface: mod_ssl/2.2.3, Library: OpenSSL/0.9.8e-fips-rhel5 [Fri Aug 05 17:46:26 2011] [debug] util_ldap.c(2021): LDAP merging Shared Cache conf: shm=0x2b22f0365ac0 rmm=0x2b22f0365b18 for VHOST: [redacted] [Fri Aug 05 17:46:26 2011] [debug] util_ldap.c(2021): LDAP merging Shared Cache conf: shm=0x2b22f0365ac0 rmm=0x2b22f0365b18 for VHOST: [redacted] [Fri Aug 05 17:46:26 2011] [info] APR LDAP: Built with OpenLDAP LDAP SDK [Fri Aug 05 17:46:26 2011] [info] LDAP: SSL support available [...] [Fri Aug 05 17:49:50 2011] [debug] mod_authnz_ldap.c(390): [client 10.8.209.142] [25726] auth_ldap authenticate: using URL ldaps://[redacted]?sAMAccountName?sub?(objectClass=user) [Fri Aug 05 17:49:50 2011] [debug] mod_authnz_ldap.c(390): [client 10.8.209.142] [25726] auth_ldap authenticate: using URL ldaps://[redacted]?sAMAccountName?sub?(objectClass=user) [Fri Aug 05 17:49:50 2011] [debug] mod_authnz_ldap.c(390): [client 10.8.209.142] [25726] auth_ldap authenticate: using URL ldaps://[redacted]?sAMAccountName?sub?(objectClass=user) [Fri Aug 05 17:49:50 2011] [debug] mod_authnz_ldap.c(390): [client 10.8.209.142] [25726] auth_ldap authenticate: using URL ldaps://[redacted]?sAMAccountName?sub?(objectClass=user) [Fri Aug 05 17:49:50 2011] [debug] mod_authnz_ldap.c(390): [client 10.8.209.142] [25726] auth_ldap authenticate: using URL ldaps://[redacted]?sAMAccountName?sub?(objectClass=user) [Fri Aug 05 17:49:50 2011] [debug] mod_authnz_ldap.c(390): [client 10.8.209.142] [25726] auth_ldap authenticate: using URL ldaps://[redacted]?sAMAccountName?sub?(objectClass=user) [Fri Aug 05 17:49:50 2011] [debug] mod_authnz_ldap.c(390): [client 10.8.209.142] [25726] auth_ldap authenticate: using URL ldaps://[redacted]?sAMAccountName?sub?(objectClass=user) [Fri Aug 05 17:49:50 2011] [warn] [client 10.8.209.142] [25726] auth_ldap authenticate: user drspruel authentication failed; URI /hg/sandbox [LDAP: ldap_simple_bind_s() failed][Can't contact LDAP server] This is the LDAP configuration in effect: <Location /hg> AuthType Basic AuthName "Mercurial LDAP Auth" AuthBasicProvider ldap AuthLDAPURL "ldaps://[redacted]?sAMAccountName?sub?(objectClass=user)" AuthLDAPBindDN '[redacted]' AuthLDAPBindPassword '[redacted]' Require ldap-group [redacted] </Location> I know the server certificate on the LDAPS is self signed so I'm setting 'LDAPVerifyServerCert Off' at the server level. Am I right in thinking that this is equivalent in nature to setting 'TLS_REQCERT' to 'never' in /etc/openldap/ldap.conf? I believe I've eliminated the LDAP (mod_ldap, mod_authnz_ldap) configuration from the picture (identical to a correctly authenticating server). I took a packet capture and based on analysis of the LDAPS communication I believe SSL/TLS trust issues are a non-factor as well at this point. I've validated that changing AuthLDAPURL to an 'ldap' connection operates correctly. The failure only triggers when using SSL. The below packet logs show that when the Apache server attempts to bind to LDAPS, it successfully establishes the TCP connection to port 636 (syn, syn-ack, ack) and then immediately tears down the connection (fin-ack, ack, fin-ack, ack). This cycle repeats 7 times in extremely quick succession (0.01 s) with no higher-layer payload being transferred; the Apache server does not even move into SSL/TLS negotiation. The 7 connect => teardown actions seem to correspond to the 7 log events. The final log message "Can't contact LDAP server" is ironic given that the Apache server itself does not go to SSL and initiates the connection teardown instead. 280 8.846495 10.29.76.11 -> 10.30.19.20 TCP 37243 > ldaps [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=4191458180 TSER=0 WS=7 281 8.847413 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37243 [SYN, ACK] Seq=0 Ack=1 Win=16384 Len=0 MSS=1460 WS=0 TSV=0 TSER=0 282 8.847423 10.29.76.11 -> 10.30.19.20 TCP 37243 > ldaps [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=4191458181 TSER=0 283 8.847825 10.29.76.11 -> 10.30.19.20 TCP 37243 > ldaps [FIN, ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=4191458181 TSER=0 284 8.848136 10.29.76.11 -> 10.30.19.20 TCP 37244 > ldaps [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=4191458182 TSER=0 WS=7 285 8.848724 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37243 [ACK] Seq=1 Ack=2 Win=65535 Len=0 TSV=16641559 TSER=4191458181 286 8.848743 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37243 [FIN, ACK] Seq=1 Ack=2 Win=65535 Len=0 TSV=16641559 TSER=4191458181 287 8.848750 10.29.76.11 -> 10.30.19.20 TCP 37243 > ldaps [ACK] Seq=2 Ack=2 Win=5888 Len=0 TSV=4191458182 TSER=16641559 288 8.849076 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37244 [SYN, ACK] Seq=0 Ack=1 Win=16384 Len=0 MSS=1460 WS=0 TSV=0 TSER=0 289 8.849083 10.29.76.11 -> 10.30.19.20 TCP 37244 > ldaps [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=4191458183 TSER=0 290 8.849209 10.29.76.11 -> 10.30.19.20 TCP 37244 > ldaps [FIN, ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=4191458183 TSER=0 291 8.849398 10.29.76.11 -> 10.30.19.20 TCP 37245 > ldaps [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=4191458183 TSER=0 WS=7 292 8.850106 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37244 [ACK] Seq=1 Ack=2 Win=65535 Len=0 TSV=16641559 TSER=4191458183 293 8.850134 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37244 [FIN, ACK] Seq=1 Ack=2 Win=65535 Len=0 TSV=16641559 TSER=4191458183 294 8.850138 10.29.76.11 -> 10.30.19.20 TCP 37244 > ldaps [ACK] Seq=2 Ack=2 Win=5888 Len=0 TSV=4191458184 TSER=16641559 295 8.850306 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37245 [SYN, ACK] Seq=0 Ack=1 Win=16384 Len=0 MSS=1460 WS=0 TSV=0 TSER=0 296 8.850312 10.29.76.11 -> 10.30.19.20 TCP 37245 > ldaps [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=4191458184 TSER=0 297 8.850451 10.29.76.11 -> 10.30.19.20 TCP 37245 > ldaps [FIN, ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=4191458184 TSER=0 298 8.850642 10.29.76.11 -> 10.30.19.20 TCP 37246 > ldaps [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=4191458184 TSER=0 WS=7 299 8.851365 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37245 [ACK] Seq=1 Ack=2 Win=65535 Len=0 TSV=16641559 TSER=4191458184 300 8.851389 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37245 [FIN, ACK] Seq=1 Ack=2 Win=65535 Len=0 TSV=16641559 TSER=4191458184 301 8.851393 10.29.76.11 -> 10.30.19.20 TCP 37245 > ldaps [ACK] Seq=2 Ack=2 Win=5888 Len=0 TSV=4191458185 TSER=16641559 302 8.851539 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37246 [SYN, ACK] Seq=0 Ack=1 Win=16384 Len=0 MSS=1460 WS=0 TSV=0 TSER=0 303 8.851546 10.29.76.11 -> 10.30.19.20 TCP 37246 > ldaps [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=4191458185 TSER=0 304 8.851663 10.29.76.11 -> 10.30.19.20 TCP 37246 > ldaps [FIN, ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=4191458185 TSER=0 305 8.852046 10.29.76.11 -> 10.30.19.20 TCP 37247 > ldaps [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=4191458186 TSER=0 WS=7 306 8.852615 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37246 [ACK] Seq=1 Ack=2 Win=65535 Len=0 TSV=16641559 TSER=4191458185 307 8.852630 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37246 [FIN, ACK] Seq=1 Ack=2 Win=65535 Len=0 TSV=16641559 TSER=4191458185 308 8.852635 10.29.76.11 -> 10.30.19.20 TCP 37246 > ldaps [ACK] Seq=2 Ack=2 Win=5888 Len=0 TSV=4191458186 TSER=16641559 309 8.853042 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37247 [SYN, ACK] Seq=0 Ack=1 Win=16384 Len=0 MSS=1460 WS=0 TSV=0 TSER=0 310 8.853050 10.29.76.11 -> 10.30.19.20 TCP 37247 > ldaps [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=4191458187 TSER=0 311 8.853204 10.29.76.11 -> 10.30.19.20 TCP 37247 > ldaps [FIN, ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=4191458187 TSER=0 312 8.853448 10.29.76.11 -> 10.30.19.20 TCP 37248 > ldaps [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=4191458187 TSER=0 WS=7 313 8.854119 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37247 [ACK] Seq=1 Ack=2 Win=65535 Len=0 TSV=16641559 TSER=4191458187 314 8.854144 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37247 [FIN, ACK] Seq=1 Ack=2 Win=65535 Len=0 TSV=16641559 TSER=4191458187 315 8.854148 10.29.76.11 -> 10.30.19.20 TCP 37247 > ldaps [ACK] Seq=2 Ack=2 Win=5888 Len=0 TSV=4191458188 TSER=16641559 316 8.854357 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37248 [SYN, ACK] Seq=0 Ack=1 Win=16384 Len=0 MSS=1460 WS=0 TSV=0 TSER=0 317 8.854362 10.29.76.11 -> 10.30.19.20 TCP 37248 > ldaps [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=4191458188 TSER=0 318 8.854501 10.29.76.11 -> 10.30.19.20 TCP 37248 > ldaps [FIN, ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=4191458188 TSER=0 319 8.854740 10.29.76.11 -> 10.30.19.20 TCP 37249 > ldaps [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=4191458188 TSER=0 WS=7 320 8.855468 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37248 [ACK] Seq=1 Ack=2 Win=65535 Len=0 TSV=16641559 TSER=4191458188 321 8.855493 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37248 [FIN, ACK] Seq=1 Ack=2 Win=65535 Len=0 TSV=16641559 TSER=4191458188 322 8.855498 10.29.76.11 -> 10.30.19.20 TCP 37248 > ldaps [ACK] Seq=2 Ack=2 Win=5888 Len=0 TSV=4191458189 TSER=16641559 323 8.855708 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37249 [SYN, ACK] Seq=0 Ack=1 Win=16384 Len=0 MSS=1460 WS=0 TSV=0 TSER=0 324 8.855713 10.29.76.11 -> 10.30.19.20 TCP 37249 > ldaps [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=4191458189 TSER=0 325 8.855859 10.29.76.11 -> 10.30.19.20 TCP 37249 > ldaps [FIN, ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=4191458189 TSER=0 326 8.856771 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37249 [ACK] Seq=1 Ack=2 Win=65535 Len=0 TSV=16641559 TSER=4191458189 327 8.856806 10.30.19.20 -> 10.29.76.11 TCP ldaps > 37249 [FIN, ACK] Seq=1 Ack=2 Win=65535 Len=0 TSV=16641559 TSER=4191458189 328 8.856811 10.29.76.11 -> 10.30.19.20 TCP 37249 > ldaps [ACK] Seq=2 Ack=2 Win=5888 Len=0 TSV=4191458190 TSER=16641559 The functioning server does not do this of course. It negotiates SSL, searches, binds and auths the user correctly. There are slight package version differences in Linux kernel, OpenLDAP and Apache. Ironically the server with the older package versions is the one authing correctly. # failing to authenticate # httpd-2.2.3-45.0.1.el5_6.1.x86_64 # openldap-2.3.43-12.el5_6.7.x86_64 # openldap-2.3.43-12.el5_6.7.i386 # Linux [redacted] 2.6.18-238.9.1.0.1.el5 #1 SMP Tue Apr 12 19:58:35 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux Server version: Apache/2.2.3 Server built: May 4 2011 05:48:12 Server's Module Magic Number: 20051115:3 Server loaded: APR 1.2.7, APR-Util 1.2.7 Compiled using: APR 1.2.7, APR-Util 1.2.7 Architecture: 64-bit Server MPM: Prefork threaded: no forked: yes (variable process count) Server compiled with.... -D APACHE_MPM_DIR="server/mpm/prefork" -D APR_HAS_SENDFILE -D APR_HAS_MMAP -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) -D APR_USE_SYSVSEM_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT -D APR_HAS_OTHER_CHILD -D AP_HAVE_RELIABLE_PIPED_LOGS -D DYNAMIC_MODULE_LIMIT=128 -D HTTPD_ROOT="/etc/httpd" -D SUEXEC_BIN="/usr/sbin/suexec" -D DEFAULT_PIDLOG="run/httpd.pid" -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" -D DEFAULT_LOCKFILE="logs/accept.lock" -D DEFAULT_ERRORLOG="logs/error_log" -D AP_TYPES_CONFIG_FILE="conf/mime.types" -D SERVER_CONFIG_FILE="conf/httpd.conf" # properly authenticating # httpd-2.2.3-22.0.1.el5.x86_64 # openldap-2.3.43-3.el5.x86_64 # openldap-2.3.43-3.el5.i386 # Linux [redacted] 2.6.18-128.1.10.0.1.el5 #1 SMP Thu May 7 17:49:20 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux Server version: Apache/2.2.3 Server built: Jan 21 2009 20:31:52 Server's Module Magic Number: 20051115:3 Server loaded: APR 1.2.7, APR-Util 1.2.7 Compiled using: APR 1.2.7, APR-Util 1.2.7 Architecture: 64-bit Server MPM: Prefork threaded: no forked: yes (variable process count) Server compiled with.... -D APACHE_MPM_DIR="server/mpm/prefork" -D APR_HAS_SENDFILE -D APR_HAS_MMAP -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) -D APR_USE_SYSVSEM_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT -D APR_HAS_OTHER_CHILD -D AP_HAVE_RELIABLE_PIPED_LOGS -D DYNAMIC_MODULE_LIMIT=128 -D HTTPD_ROOT="/etc/httpd" -D SUEXEC_BIN="/usr/sbin/suexec" -D DEFAULT_PIDLOG="logs/httpd.pid" -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" -D DEFAULT_LOCKFILE="logs/accept.lock" -D DEFAULT_ERRORLOG="logs/error_log" -D AP_TYPES_CONFIG_FILE="conf/mime.types" -D SERVER_CONFIG_FILE="conf/httpd.conf" Any ideas? -- Darren Spruell phatbuckett@xxxxxxxxx --------------------------------------------------------------------- The official User-To-User support forum of the Apache HTTP Server Project. See <URL:http://httpd.apache.org/userslist.html> for more info. To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx " from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx