Hi,
I have a Apache http server and a Tomcat server configured using AJP connector (mod_proxy_ajp). The http server serves HTML/PHP pages and also proxies Java webapp requests to back-end Tomcat server.
System config:
* Both servers are running as Virtual Machines
* CentOS 5.4
* Apache Tomcat 6.0.26 and Sun JDK 1.6
* Apache http: 2.2.3 (it's old, but that's what comes with default CentOS repo)
We are getting HTTP 500/503 errors quite frequently, but they appear to be random in nature and hence difficult to debug. I have tried increasing following config parameters, but it didn't help:
* On http server: max and smax connections in proxy_ajp.conf
* On Tomcat server: maxThreads and maxSpareThreads in Tomcat's server.xml
Following logs were seen on both sides:
== Apache http server ==
* In browser: 500/503 - internal server error.
* In Apache logs at debug mode - same logs for 500 and 503 error:
{{{
[Wed Sep 15 22:50:09 2010] [debug] ajp_header.c(430): ajp_marshal_into_msgb: Done
[Wed Sep 15 22:50:09 2010] [debug] mod_proxy_ajp.c(239): proxy: APR_BUCKET_IS_EOS
[Wed Sep 15 22:50:09 2010] [debug] mod_proxy_ajp.c(244): proxy: data to read (max 8186 at 4)
[Wed Sep 15 22:50:09 2010] [debug] mod_proxy_ajp.c(259): proxy: got 0 bytes of data
[Wed Sep 15 22:50:09 2010] [error] ajp_read_header: ajp_ilink_receive failed
[Wed Sep 15 22:50:09 2010] [error] (120006)APR does not understand this error code: proxy: read response failed from 10.0.0.10:9080 (
tomcat.lab.uab.edu)
[Wed Sep 15 22:50:09 2010] [debug] proxy_util.c(2062): proxy: AJP: has released connection for (
tomcat.lab.uab.edu)
[Wed Sep 15 22:50:09 2010] [debug] ssl_engine_kernel.c(1765): OpenSSL: Write: SSL negotiation finished successfully
[Wed Sep 15 22:50:09 2010] [info] [client 10.0.0.8] Connection closed to child 3 with standard shutdown (server httpd.lab.uab.edu:443)
}}}
* tcpdump on Apache http server: Shows a packet going out to Tomcat server
== Tomcat server ==
* In Tomcat logs catalina.out: No logs
* In Tomcat access logs catalina.out: No logs
* tcpdump on Tomcat: No incoming packet seen
The request doen't seem to reach Tomcat at all. I have checked syslog (/var/log/messages), but I don't see any dropped packet logs as well.
Anyone else had this issues before? Any pointers on how to debugs this would be really helpful.
Thanks,
Shantanu Pavgi.
