I was able to reproduce with ab by requesting a php script that has an infinite loop. Running multiple copies of this script concurrently brings httpd into a hang state, that it will not recover from without manually restarting the httpd server. ab -n 300 -c 300 http://myserver/mysite/loop.php <?php /* loop.php */ while (1) sleep (20); ?> I have timeout set to 900 but it does not appear to be working. Will this timeout protect the server from such scripts or should is there something else. I am configured using Prefork settings are: Timeout 900 MaxKeepAliveRequests 100 StartServers 5 MinSpareServers 5 MaxSpareServers 10 MaxClients 256 MaxRequestsPerChild 0 truss of root is always:: pollsys(0xFFBFE388, 0, 0xFFBFE3F0, 0x00000000) = 0 waitid(P_ALL, 0, 0xFFBFE310, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) = 0 pollsys(0xFFBFE388, 0, 0xFFBFE3F0, 0x00000000) = 0 waitid(P_ALL, 0, 0xFFBFE310, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) = 0 pollsys(0xFFBFE388, 0, 0xFFBFE3F0, 0x00000000) = 0 waitid(P_ALL, 0, 0xFFBFE310, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) = 0 pollsys(0xFFBFE388, 0, 0xFFBFE3F0, 0x00000000) = 0 waitid(P_ALL, 0, 0xFFBFE310, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) = 0 pollsys(0xFFBFE388, 0, 0xFFBFE3F0, 0x00000000) = 0 waitid(P_ALL, 0, 0xFFBFE310, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) = 0 pollsys(0xFFBFE388, 0, 0xFFBFE3F0, 0x00000000) = 0 waitid(P_ALL, 0, 0xFFBFE310, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) = 0 pollsys(0xFFBFE388, 0, 0xFFBFE3F0, 0x00000000) = 0 waitid(P_ALL, 0, 0xFFBFE310, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) = 0 pollsys(0xFFBFE388, 0, 0xFFBFE3F0, 0x00000000) = 0 waitid(P_ALL, 0, 0xFFBFE310, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) = 0 pollsys(0xFFBFE388, 0, 0xFFBFE3F0, 0x00000000) = 0 waitid(P_ALL, 0, 0xFFBFE310, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) = 0 pollsys(0xFFBFE388, 0, 0xFFBFE3F0, 0x00000000) = 0 waitid(P_ALL, 0, 0xFFBFE310, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) = 0 pollsys(0xFFBFE388, 0, 0xFFBFE3F0, 0x00000000) = 0 waitid(P_ALL, 0, 0xFFBFE310, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) = 0 # pstack 22040 (root) 22040: /usr/local/apache/bin/httpd -d /usr/local/apache -f /usr/local/apache/ feccc4a8 pollsys (ffbfe388, 0, ffbfe3f0, 0) fec67dc0 pselect (ffbfe388, fed34660, fed34660, 0, ffbfe3f0, 0) + 1c8 fec68138 select (0, 0, 0, 0, ffbfe458, 0) + a0 ff05f5bc apr_sleep (0, f4240, ffbfe554, 0, d0b78, 11176) + 4c 00048fa8 ap_wait_or_timeout (ffbfe554, ffbfe550, ffbfe5d8, d0b78, c5c00, c2800) + 60 00087530 ap_mpm_run (c5c00, c3800, ff, c5c00, c6000, c5c00) + 33c 0002e1e0 main (d0b78, c0c00, c3000, c3000, ceb70, 0) + 784 0002d57c _start (0, 0, 0, 0, 0, 0) + 5c pstack of worker is revealing: pstack 4851 4851: /usr/local/apache/bin/httpd -d /usr/local/apache -f /usr/local/apache/ feccbad0 nanosleep (ffbfd8c0, ffbfd8b8) fe63df54 zif_sleep (1, 1bfaa0, 0, 0, 0, ffbfda60) + 54 fe719330 zend_do_fcall_common_helper_SPEC (ffbfda78, 1, 0, 0, 18, 0) + 940 fe7188f8 execute (1bf580, 1bf618, 1d, ffbfdb44, 4, fe7187ec) + 10c fe6f8bbc zend_execute_scripts (8, 0, 3, ffbfdbb0, fe8a5ed0, ffbfe0f8) + d4 fe6b11e8 php_execute_script (0, fe851228, 6, 0, 70687000, 0) + 218 fe793064 php_handler (2cfbc0, 0, 2cfbc0, c2000, c32b0, 9) + 2cc 00041014 ap_run_handler (2cfbc0, 3b3b3b3b, 70687000, 80808080, ff00, 80808080) + 3c 00041490 ap_invoke_handler (165870, 96c00, 2cfbc0, ffbfe254, fe930028, 0) + b8 0006c310 ap_process_request (2cfbc0, 0, 4, 2cfbc0, 0, 0) + 160 00069478 ap_process_http_connection (2c9e40, 2c9ba8, 2c9ba8, fe, c3b10, eaf60) + 10c 000477b8 ap_run_process_connection (2c9e40, 2c9ba8, 2c9ba8, fe, 2c7bf0, 2cdb78) + 3c 00086ea4 child_main (0, 1, c6000, c5c00, 11177, c5c00) + 42c 00087108 make_child (86800, fe, 6, 0, d66d0, c5c00) + ec 00087b10 ap_mpm_run (c5c00, c3800, ee, c5c00, c6000, c5c00) + 91c 0002e1e0 main (d0b78, c0c00, c3000, c3000, ceb70, 0) + 784 0002d57c _start (0, 0, 0, 0, 0, 0) + 5c # truss -p 4851 nanosleep(0xFFBFD8C0, 0xFFBFD8B8) (sleeping...) nanosleep(0xFFBFD8C0, 0xFFBFD8B8) = 0 nanosleep(0xFFBFD8C0, 0xFFBFD8B8) (sleeping...) ps -ef | grep /usr/local/apache/bin/httpd | wc -l 258 these scripts have been tying up my server for hours, so it is clear Timeout 900 is not working. If anyone can Let me know if there is something better. and why Timeout does not work for this (and what it is used for). I would be most thankful. Thank You. Warren From: Jeroen Geilman [mailto:jeroen@xxxxxxxxx] On 06/16/2011 05:30 PM, Zaccone, Warren wrote: I ran -S and it showed that I have port :443 only in addition to 80. It is a simple setup. 99% of load is on port 80. server-status is not revealing anything,. it shows the last requests that were successful. It appears that the httpd root process is no longer responding yet is running. When you say it could be lots of things. Please let me know some ideas to try. Could it be a configure/build issue or related to operating system patches? (I built Apache 2.2.15 on Solaris 10).
thank you. From: Jeroen Geilman [mailto:jeroen@xxxxxxxxx] On 06/03/2011 10:12 PM, Zaccone, Warren wrote: Have a problem where httpd server 2.2.15 stops responding to requests requiring server to be frequently restarted that has me going in circles. I was looking for some direction as to how to pursue. there are 8 workers running each consuming very little cpu. netstat shows process listening on port 80 and 443. requests on 443 are served fine, but requests on 80 hang, Backed out 2.2.15 and went to previous release I had built (httpd 2.2.11) and problem has gone away. No other variables changed, and they were built same way, so I am wondering if there was a change in behavior between the releases that I have not anticipated or if there is a bug that may have been fixed subsequently. I am testing 2.2.19 in my lab without issues, but 2.2.15 is in production so I need to determine the cause. the requests are 99% php scripts with a fair number using web services with nusoap. However I think the issue may be httpd itself because port 443 works fine, but port 80 does not respond. Initially both ports are functioning and over time, (a few hours), requests on port 80 (http) stop responding but 443 (https) remains fine. restarting httpd fixes it for a few hours. I appreciate any thoughts or direction. thank you. Warren
I compiled it as Apache/2.2.15 (Unix) PHP/5.2.14 mod_ssl/2.2.15 OpenSSL/0.9.8o apachectl -V
-- J. -- J. |
|