Re: [users@httpd] Bloated Processes won't die

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Preston Podaril wrote:
Running Apache 1.3.33 (compiled with mod_ssl, uses mod_gzip and mod_auth_mysql also), php 4.3.10, phpa 1.3.3r2, mysql 4.0.18 on remote server. the problem exists on the servers not actually starting apache with SSL, just /etc/init.d/apache start. These are running in Solaris 8 on sparc hardware.

I have apache processes that will bloat up to over 100 MB, I have seen as high as 125 MB, or RES memory according to top. Even such a large number that I see between 50M and 80 MB is ridiculous, also it seems strange to me that the server would be using so much swap when it has 1/2 a GB of real memory free...:

load averages:  1.18,  0.83,  0.83                10:35:54
68 processes:  67 sleeping, 1 on cpu
CPU states: 62.5% idle, 23.2% user, 14.4% kernel,  0.0% iowait,  0.0% swap
Memory: 2048M real, 547M free, 1293M swap in use, 2198M swap free

   PID USERNAME THR PRI NICE  SIZE   RES STATE    TIME    CPU COMMAND
 20354 nobody     3  31    0  221M  101M sleep    2:27  3.71% libhttpd.ep
 20408 nobody     3  58    0  155M   38M sleep    0:24  0.00% libhttpd.ep
 20318 nobody     3  60    0  155M   36M sleep    0:25  0.00% libhttpd.ep

I performed an lsof -p 20354 to see what all it had open and seen this:

COMMAND     PID   USER   FD   TYPE        DEVICE SIZE/OFF   NODE NAME
libhttpd. 20354 nobody  cwd   VDIR         136,0     1024      2 /
libhttpd. 20354 nobody txt VREG 253,1 30676 349796 /usr/local/etc/apache-1.3.33/libexec/libhttpd.ep libhttpd. 20354 nobody txt VREG 136,0 44892 98711 /usr/lib/nss_files.so.1 libhttpd. 20354 nobody txt VREG 136,0 24536 98707 /usr/lib/nss_dns.so.1 libhttpd. 20354 nobody txt VREG 253,1 163532 349808 /usr/local/etc/apache-1.3.33/libexec/php_accelerator.so libhttpd. 20354 nobody txt VREG 136,0 382600 98706 /usr/lib/libresolv.so.2 libhttpd. 20354 nobody txt VREG 253,1 1305976 445372 /usr/local/ssl/lib/libcrypto.so.0.9.7 libhttpd. 20354 nobody txt VREG 253,1 232068 445376 /usr/local/ssl/lib/libssl.so.0.9.7 libhttpd. 20354 nobody txt VREG 253,1 10581944 349801 /usr/local/etc/apache-1.3.33/libexec/libphp4.so libhttpd. 20354 nobody txt VREG 136,0 225712 98703 /usr/lib/libldap.so.4 libhttpd. 20354 nobody txt VREG 136,0 46936 98469 /usr/lib/libaio.so.1 libhttpd. 20354 nobody txt VREG 253,1 760864 390565 /usr/local/lib/libgcc_s.so.1 libhttpd. 20354 nobody txt VREG 136,0 42184 98472 /usr/lib/libgen.so.1 libhttpd. 20354 nobody txt VREG 136,0 16020 98459 /usr/lib/libcrypt_i.so.1 libhttpd. 20354 nobody txt VREG 136,0 41816 98677 /usr/lib/librt.so.1 libhttpd. 20354 nobody txt VREG 253,1 78980 389700 /usr/local/lib/libz.so.1.2.1 libhttpd. 20354 nobody txt VREG 136,0 102424 98536 /usr/lib/libm.so.1 libhttpd. 20354 nobody txt VREG 253,1 161544 406192 /usr/local/etc/mysql/lib/mysql/libmysqlclient.so.12.0.0 libhttpd. 20354 nobody txt VREG 253,1 70520 349799 /usr/local/etc/apache-1.3.33/libexec/mod_auth_mysql.so libhttpd. 20354 nobody txt VREG 253,1 109228 349800 /usr/local/etc/apache-1.3.33/libexec/mod_gzip.so libhttpd. 20354 nobody txt VREG 136,0 191996 98723 /usr/lib/libthread.so.1 libhttpd. 20354 nobody txt VREG 136,0 1146168 98718 /usr/lib/libc.so.1 libhttpd. 20354 nobody txt VREG 136,0 17096 155652 /usr/platform/sun4u/lib/libc_psr.so.1 libhttpd. 20354 nobody txt VREG 136,0 24968 98487 /usr/lib/libmp.so.2 libhttpd. 20354 nobody txt VREG 136,0 898580 98721 /usr/lib/libnsl.so.1 libhttpd. 20354 nobody txt VREG 136,0 38900 98722 /usr/lib/libpthread.so.1 libhttpd. 20354 nobody txt VREG 136,0 70864 98657 /usr/lib/libsocket.so.1 libhttpd. 20354 nobody txt VREG 253,1 1005004 349797 /usr/local/etc/apache-1.3.33/libexec/libhttpd.so libhttpd. 20354 nobody txt VREG 136,0 5296 98495 /usr/lib/libdl.so.1 libhttpd. 20354 nobody txt VREG 136,0 227520 98479 /usr/lib/ld.so.1 libhttpd. 20354 nobody 0r VCHR 13,2 0t0 172107 /devices/pseudo/mm@0:null libhttpd. 20354 nobody 1w VCHR 13,2 0t0 172107 /devices/pseudo/mm@0:null libhttpd. 20354 nobody 2w VREG 136,3 14979 44904 /export/apache/logs/error_log libhttpd. 20354 nobody 3u VREG 136,3 0 44822 /export/apache/logs/httpd.mm.22204.sem libhttpd. 20354 nobody 15w VREG 136,3 14979 44904 /export/apache/logs/error_log libhttpd. 20354 nobody 16u IPv4 0x30007e22550 0t0 TCP *:80 (LISTEN) libhttpd. 20354 nobody 17w VREG 136,3 1409414 44959 /export/apache/logs/access_log libhttpd. 20354 nobody 18w VREG 136,3 0 44960 /export (/dev/dsk/c0t0d0s3)

I have tried trussing the process also to see if I could notice anything, but I am not even sure what to look for in there.

These larger processes sometimes will go into a perpetual sleep also, and will never close, but never get anymore cpu time. It happens some times but not always. They have to be killed manually with a kill -9. A lot of times on those particular process and lsof shows an Established tcp connection to the mysql server.

My httpd.conf is setup like this to start:

ServerType standalone
ServerRoot "/usr/local/etc/apache"
LockFile /export/apache/logs/httpd.lock
PidFile /export/apache/logs/httpd.pid
ScoreBoardFile /export/apache/logs/httpd.scoreboard
Timeout 300
KeepAlive On
MaxKeepAliveRequests 1000
KeepAliveTimeout 5
MinSpareServers 10
MaxSpareServers 20
StartServers 20
MaxClients 150
MaxRequestsPerChild 100
LoadModule gzip_module        libexec/mod_gzip.so
LoadModule mysql_auth_module  libexec/mod_auth_mysql.so
LoadModule php4_module        libexec/libphp4.so
<IfDefine SSL>
LoadModule ssl_module         libexec/libssl.so
</IfDefine>

I am at a loss right now of where to go next, if anyone has any insight or ideas on what I can do to track this down, it would be much appreciated.

Preston Podaril
ppod@xxxxxxxxxxxxxxxxxxx
Network Administrator
New Digital Group

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx
  "   from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx


I also compiled in and we are using OSSP mm shared memory allocation 1.30 if this helps anyone come up with any ideas. Or some tools I can use to dig into this apparent Memory leak.

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx
  "   from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx



[Index of Archives]     [Open SSH Users]     [Linux ACPI]     [Linux Kernel]     [Linux Laptop]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Squid]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Video 4 Linux]     [Device Mapper]

  Powered by Linux