-bash-3.2$ uname -a
Linux hdm102 2.6.18-238.19.1.el5.centos.plus #1 SMP Mon Jul 18 10:05:09 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux
-bash-3.2$ cat /etc/issue
CentOS release 5.6 (Final)
Kernel \r on an \m
-bash-3.2$ ./httpd -V
Server version: Apache/2.2.20 (Unix)
Server built: Sep 8 2011 23:41:28
Server's Module Magic Number: 20051115:28
Server loaded: APR 1.4.5, APR-Util
1.3.12
Compiled using: APR 1.4.5, APR-Util 1.3.12
Architecture: 64-bit
Server MPM: Worker
threaded: yes (fixed thread count)
forked: yes (variable process count)
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/worker"
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=128
-D HTTPD_ROOT="/usr/local/apache2"
-D SUEXEC_BIN="/usr/local/apache2/bin/suexec"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"
I have a custom module(mod_XXXX.so) and oracle weblogic apache plugin(mod_wl_22.so) loaded via LoadModule directive into apache.
The load that is being tested is 2500 req/second.
Under this load apache continues to run for 2-3 days but suddnely stops responding. Has been happening consistently.
Once this condition occurs apache stops serving even static pages exa: index.html or does not display server-status(mod_status).
And the only way to recover is to restart the server.
The errors that I see in the error log are the following ones, which keep showing up frequently of which the first one when the weblogic server refuses connections dues to overload condition.
1. [Tue Sep 20
08:15:11 2011] [error] [client 135.115.142.108] ap_proxy: trying POST /cwmp-allow/cwmpWeb/CPEMgt at backend host '64.186.185.155/7003; got exception 'CONNECTION_REFUSED [os error=0, line 1715 of ../nsapi/URL.cpp]: Error connecting to host 64.186.185.155:7003'
2. [Tue Sep 20 08:15:12 2011] [error] [client 135.115.142.100] (34)Numerical result out of range: mod_XXXX: XXXX_throttle_handler(): failed to release the mutex lock
Had a look at the errno of apr and (34)Numerical result out of range doesn't seem to be coming from APR.
Has some come across
such a issue? Any kind of help/advice will really helpfull.
