Hi,
We are experiencing problems with Apache 2.2.9 on Windows 2000 Server
SP4. Our site handles around 100,000 hits per day.
The problems started when we upgraded from 2.0.47.
The server restarts several times a day with the following message (from
error.log):
[notice] Parent: child process exited with status 3221225477 --
Restarting.
On some occassions, httpd.exe (the child process) is using 100% CPU and
cannot be killed, and the server has to be rebooted.
The access logs for the relevant times have nothing obvious in common,
and there are no entries in the windows event logs.
We have been unable to reproduce the fault on demand.
Though there are many postings about the error message, none of them
seem to apply to our situation, because we are not using perl, python,
php, mysql, or ldap authentication.
Apache is configured to serve html pages using mod_include, and we are
using mod_proxy to pass requests to IIS for the dynamic pages.
Here are the relevant portions of the config file:
LoadModule alias_module modules/mod_alias.so
LoadModule authz_default_module modules/mod_authz_default.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule cgi_module modules/mod_cgi.so
LoadModule dir_module modules/mod_dir.so
LoadModule include_module modules/mod_include.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule mime_module modules/mod_mime.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
<Directory />
Options FollowSymLinks Includes
AllowOverride None
</Directory>
<Directory "c:/foo">
Options Indexes FollowSymLinks +Includes
AllowOverride None
Order allow,deny
Allow from all
</Directory>
ErrorLog "logs/error.log"
LogLevel warn
<IfModule log_config_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\"
\"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
<IfModule logio_module>
# You need to enable mod_logio.c to use %I and %O
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\"
\"%{User-Agent}i\" %I %O" combinedio
</IfModule>
CustomLog "logs/access.log" common
</IfModule>
<Directory "C:/Apache2.2/cgi-bin">
AllowOverride None
Options None
Order allow,deny
Allow from all
</Directory>
DefaultType text/plain
<IfModule mime_module>
TypesConfig conf/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
AddOutputFilter INCLUDES .html
</IfModule>
#Added because of error: winnt_accept: Asynchronous AcceptEx failed. in
error.log
EnableMMAP off
EnableSendfile off
Win32DisableAcceptEx
#Added because of error: [warn] Server ran out of threads to serve
requests. Consider raising the ThreadsPerChild setting
<IfModule mpm_winnt_module>
ThreadsPerChild 250
MaxRequestsPerChild 0
</IfModule>
AddCharset UTF-8 .utf8
AddDefaultCharset UTF-8
#IP addresses, ports, and file locations have been anonymized.
<VirtualHost 1.2.3.4:80>
ServerName www.dummy-host.com
ServerAdmin webmaster@xxxxxxxxxxxxxxxxxxxxxx
DocumentRoot "c:/foo"
DirectoryIndex index.html "/site_files/pages/Home.html"
ErrorLog C:/Logs/foo-error.log
CustomLog "|C:/Logs/bin/rotatelogs.exe
C:/Logs/foo-access_%Y%m%d.log 86400" combined
ProxyPass /someservice/ http://1.2.3.4:1234/
ProxyPass /someotherservice/ http://1.2.3.4:2345/
<directory "c:/foo/redirectme">
RedirectMatch 301 /.*
http://www.dummy-host.com/redirectmehere.html
</directory>
</VirtualHost>
NameVirtualHost 1.2.3.5:80
<VirtualHost 1.2.3.5:80>
ServerName www.anotherdummyhost.com
ServerAdmin webmaster@xxxxxxxxxxxxxxxxxxxxxx
ErrorLog C:/Logs/bar-error.log
CustomLog "|C:/Logs/bin/rotatelogs.exe
C:/Logs/bar-access_%Y%m%d.log 86400" combined
ProxyPass /someservice/ http://1.2.3.5:1234/
ProxyPass /someotherservice/ http://1.2.3.5:2345/
<Proxy *>
Order Deny,Allow
Allow from all
</Proxy>
</VirtualHost>
<VirtualHost 1.2.3.5:80>
ServerName www.yetanotherdummyhost.com
ServerAdmin webmaster@xxxxxxxxxxxxxxxxxxxxxx
ErrorLog C:/Logs/foobar-error.log
CustomLog "|C:/Logs/bin/rotatelogs.exe
C:/Logs/foobar-access_%Y%m%d.log 86400" combined
RedirectMatch permanent .*
http://www.somelocalsite.com/redirectmehere.html
</VirtualHost>
I got a stack back trace from the most recent crashes, and the faulting
thread details are shown below:
Application exception occurred:
App: (pid=896)
When: 19/11/2008 @ 17:41:21.875
Exception number: c0000005 (access violation)
State Dump for Thread Id 0xe88
eax=07aec068 ebx=00000000 ecx=00000000 edx=00001f50 esi=07aec068
edi=00000000
eip=6eec72df esp=0498fd1c ebp=0498fd2c iopl=0 nv up ei pl zr na
po nc
cs=001b ss=0023 ds=0023 es=0023 fs=0038 gs=0000
efl=00000246
function: apr_allocator_free
6eec72d0 55 push ebp
6eec72d1 8bec mov ebp,esp
6eec72d3 51 push ecx
6eec72d4 53 push ebx
6eec72d5 56 push esi
6eec72d6 8b750c mov esi,[ebp+0xc]
ss:04d59c12=00000000
6eec72d9 57 push edi
6eec72da 8b7d08 mov edi,[ebp+0x8]
ss:04d59c12=00000000
6eec72dd 33db xor ebx,ebx
FAULT ->6eec72df 8b470c mov eax,[edi+0xc]
ds:003c9ee6=????????
6eec72e2 85c0 test eax,eax
6eec72e4 7406 jz apr_socket_data_get+0x3c
(6eecc2ec)
6eec72e6 50 push eax
6eec72e7 e8b4f8ffff call apr_thread_mutex_lock
(6eec6ba0)
6eec72ec 8b07 mov eax,[edi]
ds:00000000=????????
6eec72ee 8b4f04 mov ecx,[edi+0x4]
ds:003c9ee6=????????
6eec72f1 8b5708 mov edx,[edi+0x8]
ds:003c9ee6=????????
6eec72f4 89450c mov [ebp+0xc],eax
ss:04d59c12=00000000
6eec72f7 894d08 mov [ebp+0x8],ecx
ss:04d59c12=00000000
6eec72fa 8b06 mov eax,[esi]
ds:07aec068=00000000
6eec72fc 8b4d08 mov ecx,[ebp+0x8]
ss:04d59c12=00000000
*----> Stack Back Trace <----*
FramePtr ReturnAd Param#1 Param#2 Param#3 Param#4 Function Name
0498FD2C 6EE61DCC 00000000 07AEC068 0498FD4C 6EE62248
!apr_allocator_free
0498FD3C 6EE62248 07AEC090 07AEA0F8 0498FD60 6EE6102B !apr_bucket_free
0498FD4C 6EE6102B 07AEA218 026FC7F8 0700C358 0498FD70
!apr_bucket_flush_create
0498FD60 6EE6105C 0700C358 00000000 0498FEDC 6FF09FF1
!apr_brigade_cleanup
0498FD70 6FF09FF1 0700C358 07003200 07003200 062F5290
!apr_brigade_destroy
0498FEDC 6FF19CC2 062F56D8 0700C358 07022378 0498FF0C
!ap_get_request_note
0498FEF0 6FF0E9DC 062F56D8 07003200 07022378 07022378 !ap_pass_brigade
0498FF0C 6FF0E933 07022378 062F5290 0498FF34 6FF0A87C !ap_die
0498FF1C 6FF0A87C 07022378 00000000 0042A6F0 062F5290 !ap_die
0498FF34 6FF04D21 062F5290 062F5290 062F5290 000000DE
!ap_get_request_note
0498FF4C 6FF04FD3 062F5290 004E3BC0 0498FF7C 6FF1D2DC
!ap_run_process_connection
0498FF5C 6FF1D2DC 062F5290 026FA780 00000000 026FA570
!ap_process_connection
0498FF7C 780085BC 062F5288 00000000 78008532 026FA570
!ap_regkey_value_remove
0498FFB4 7C57B396 026FA570 00000000 78008532 026FA570 !endthreadex
0498FFEC 00000000 00000000 00000000 00000000 00000000 kernel32!lstrcmpiW
Application exception occurred:
App: (pid=3392)
When: 19/11/2008 @ 18:06:06.796
Exception number: c0000005 (access violation)
State Dump for Thread Id 0x4fc
eax=0047c1a0 ebx=02af92d8 ecx=00000000 edx=0047c1a0 esi=004fd3c8
edi=05b0592c
eip=6ee6101e esp=03b0fd58 ebp=03b0fd60 iopl=0 nv up ei ng nz ac
po cy
cs=001b ss=0023 ds=0023 es=0023 fs=0038 gs=0000
efl=00000297
function: apr_brigade_cleanup
6ee61007 57 push edi
6ee61008 8b7004 mov esi,[eax+0x4]
ds:00846086=????????
6ee6100b 8d7804 lea edi,[eax+0x4]
ds:00846086=????????
6ee6100e 3bf7 cmp esi,edi
6ee61010 7426 jz apr_memcache_multgetp+0x368
(6ee69b38)
6ee61012 8b0e mov ecx,[esi]
ds:004fd3c8=00000000
6ee61014 8b5604 mov edx,[esi+0x4]
ds:008c72ae=00000000
6ee61017 890a mov [edx],ecx
ds:0047c1a0=00000000
6ee61019 8b4604 mov eax,[esi+0x4]
ds:008c72ae=00000000
6ee6101c 8b0e mov ecx,[esi]
ds:004fd3c8=00000000
FAULT ->6ee6101e 894104 mov [ecx+0x4],eax
ds:003c9ee6=????????
6ee61021 8b4618 mov eax,[esi+0x18]
ds:008c72ae=00000000
6ee61024 8b5608 mov edx,[esi+0x8]
ds:008c72ae=00000000
6ee61027 50 push eax
6ee61028 ff520c call dword ptr [edx+0xc]
ds:00846086=????????
6ee6102b 56 push esi
6ee6102c ff561c call dword ptr [esi+0x1c]
ds:008c72ae=00000000
6ee6102f 8b37 mov esi,[edi]
ds:05b0592c=004fd3c8
6ee61031 83c408 add esp,0x8
6ee61034 3bf7 cmp esi,edi
6ee61036 75da jnz apr_posix_perms2mode+0x1c2
(6ee67012)
6ee61038 5f pop edi
*----> Stack Back Trace <----*
FramePtr ReturnAd Param#1 Param#2 Param#3 Param#4 Function Name
03B0FD60 6EE6105C 05B05928 000B23C5 03B0FEDC 6FF09FF1
!apr_brigade_cleanup
03B0FD70 6FF09FF1 05B05928 05AFCC80 05AFCC80 05AF8930
!apr_brigade_destroy
03B0FEDC 6FF19CC2 05AF8D78 05B05928 05B07930 03B0FF0C
!ap_get_request_note
03B0FEF0 6FF0E9DC 05AF8D78 05AFCC80 05B07930 05B07930 !ap_pass_brigade
03B0FF0C 6FF0E933 05B07930 05AF8930 03B0FF34 6FF0A87C !ap_die
03B0FF1C 6FF0A87C 05B07930 00000000 0042A6F0 05AF8930 !ap_die
03B0FF34 6FF04D21 05AF8930 05AF8930 05AF8930 000000BD
!ap_get_request_note
03B0FF4C 6FF04FD3 05AF8930 004E1958 03B0FF7C 6FF1D2DC
!ap_run_process_connection
03B0FF5C 6FF1D2DC 05AF8930 02AF7200 00000000 0071F730
!ap_process_connection
03B0FF7C 780085BC 05AF8928 00000000 00000000 0071F730
!ap_regkey_value_remove
03B0FFB4 7C57B396 0071F730 00000000 00000000 0071F730 !endthreadex
03B0FFEC 00000000 00000000 00000000 00000000 00000000 kernel32!lstrcmpiW
Application exception occurred:
App: (pid=2988)
When: 19/11/2008 @ 22:38:41.250
Exception number: c0000005 (access violation)
State Dump for Thread Id 0x67c
eax=004981c0 ebx=00000000 ecx=00000000 edx=00001f50 esi=004981c0
edi=00000000
eip=6eec72df esp=03b8fd1c ebp=03b8fd2c iopl=0 nv up ei pl zr na
po nc
cs=001b ss=0023 ds=0023 es=0023 fs=0038 gs=0000
efl=00000246
function: apr_allocator_free
6eec72d0 55 push ebp
6eec72d1 8bec mov ebp,esp
6eec72d3 51 push ecx
6eec72d4 53 push ebx
6eec72d5 56 push esi
6eec72d6 8b750c mov esi,[ebp+0xc]
ss:03f59c12=????????
6eec72d9 57 push edi
6eec72da 8b7d08 mov edi,[ebp+0x8]
ss:03f59c12=????????
6eec72dd 33db xor ebx,ebx
FAULT ->6eec72df 8b470c mov eax,[edi+0xc]
ds:003c9ee6=????????
6eec72e2 85c0 test eax,eax
6eec72e4 7406 jz apr_socket_data_get+0x3c
(6eecc2ec)
6eec72e6 50 push eax
6eec72e7 e8b4f8ffff call apr_thread_mutex_lock
(6eec6ba0)
6eec72ec 8b07 mov eax,[edi]
ds:00000000=????????
6eec72ee 8b4f04 mov ecx,[edi+0x4]
ds:003c9ee6=????????
6eec72f1 8b5708 mov edx,[edi+0x8]
ds:003c9ee6=????????
6eec72f4 89450c mov [ebp+0xc],eax
ss:03f59c12=????????
6eec72f7 894d08 mov [ebp+0x8],ecx
ss:03f59c12=????????
6eec72fa 8b06 mov eax,[esi]
ds:004981c0=00000000
6eec72fc 8b4d08 mov ecx,[ebp+0x8]
ss:03f59c12=????????
*----> Stack Back Trace <----*
FramePtr ReturnAd Param#1 Param#2 Param#3 Param#4 Function Name
03B8FD2C 6EE61DCC 00000000 004981C0 03B8FD4C 6EE62248
!apr_allocator_free
03B8FD3C 6EE62248 004981E8 004FD3C8 03B8FD60 6EE6102B !apr_bucket_free
03B8FD4C 6EE6102B 004FD4E8 023817A0 07593278 03B8FD70
!apr_bucket_flush_create
03B8FD60 6EE6105C 07593278 00000000 03B8FEDC 6FF09FF1
!apr_brigade_cleanup
03B8FD70 6FF09FF1 07593278 07586110 07586110 05BB0B48
!apr_brigade_destroy
03B8FEDC 6FF19CC2 05BB0F90 07593278 07583228 03B8FF0C
!ap_get_request_note
03B8FEF0 6FF0E9DC 05BB0F90 07586110 07583228 07583228 !ap_pass_brigade
03B8FF0C 6FF0E933 07583228 05BB0B48 03B8FF34 6FF0A87C !ap_die
03B8FF1C 6FF0A87C 07583228 00000000 0042A6F0 05BB0B48 !ap_die
03B8FF34 6FF04D21 05BB0B48 05BB0B48 05BB0B48 000000BF
!ap_get_request_note
03B8FF4C 6FF04FD3 05BB0B48 004E1AF0 03B8FF7C 6FF1D2DC
!ap_run_process_connection
03B8FF5C 6FF1D2DC 05BB0B48 0237F668 00000000 023421B8
!ap_process_connection
03B8FF7C 780085BC 05BB0B40 00000000 00000000 023421B8
!ap_regkey_value_remove
03B8FFB4 7C57B396 023421B8 00000000 00000000 023421B8 !endthreadex
03B8FFEC 00000000 00000000 00000000 00000000 00000000 kernel32!lstrcmpiW
Application exception occurred:
App: (pid=952)
When: 20/11/2008 @ 09:43:00.781
Exception number: c0000005 (access violation)
State Dump for Thread Id 0xb8c
eax=646e6957 ebx=0047c270 ecx=00b4fec8 edx=00b4fecc esi=00000000
edi=0047c270
eip=6ff09c0c esp=00b4fd6c ebp=00b4fedc iopl=0 nv up ei ng nz ac
pe cy
cs=001b ss=0023 ds=0023 es=0023 fs=0038 gs=0000
efl=00000293
function: ap_get_request_note
6ff09bf4 8b4314 mov eax,[ebx+0x14]
ds:00846156=????????
6ff09bf7 8945cc mov [ebp+0xcc],eax
ss:00f19dc2=????????
6ff09bfa e932020000 jmp ap_get_request_note+0x1251
(6ff09e31)
6ff09bff 8b4dd4 mov ecx,[ebp+0xd4]
ss:00f19dc2=????????
6ff09c02 8d55f0 lea edx,[ebp+0xf0]
ss:00f19dc2=????????
6ff09c05 51 push ecx
6ff09c06 8d4dec lea ecx,[ebp+0xec]
ss:00f19dc2=????????
6ff09c09 52 push edx
6ff09c0a 51 push ecx
6ff09c0b 53 push ebx
FAULT ->6ff09c0c ff5010 call dword ptr [eax+0x10]
ds:64ab083d=????????
6ff09c0f 83c410 add esp,0x10
6ff09c12 83f80b cmp eax,0xb
6ff09c15 0f845a020000 je ap_get_request_note+0x1295
(6ff09e75)
6ff09c1b 3d68fd0a00 cmp eax,0xafd68
6ff09c20 0f844f020000 je ap_get_request_note+0x1295
(6ff09e75)
6ff09c26 3dd9fc0a00 cmp eax,0xafcd9
6ff09c2b 0f8444020000 je ap_get_request_note+0x1295
(6ff09e75)
6ff09c31 3d57fd0a00 cmp eax,0xafd57
6ff09c36 0f8439020000 je ap_get_request_note+0x1295
(6ff09e75)
6ff09c3c 3d24fd0a00 cmp eax,0xafd24
6ff09c41 0f842e020000 je ap_get_request_note+0x1295
(6ff09e75)
*----> Stack Back Trace <----*
FramePtr ReturnAd Param#1 Param#2 Param#3 Param#4 Function Name
00B4FEDC 6FF19CC2 039760C8 03978C80 03982C78 00B4FF0C
!ap_get_request_note
00B4FEF0 6FF0E9DC 039760C8 06867E90 03982C78 03982C78 !ap_pass_brigade
00B4FF0C 6FF0E933 03982C78 03975C80 00B4FF34 6FF0A87C !ap_die
00B4FF1C 6FF0A87C 03982C78 00000000 0042A6F0 03975C80 !ap_die
00B4FF34 6FF04D21 03975C80 03975C80 03975C80 0000000E
!ap_get_request_note
00B4FF4C 6FF04FD3 03975C80 00425430 00B4FF7C 6FF1D2DC
!ap_run_process_connection
00B4FF5C 6FF1D2DC 03975C80 005A4DE8 00000000 0051F810
!ap_process_connection
00B4FF7C 780085BC 03975C78 00000000 00000000 0051F810
!ap_regkey_value_remove
00B4FFB4 7C57B396 0051F810 00000000 00000000 0051F810 !endthreadex
00B4FFEC 00000000 00000000 00000000 00000000 00000000 kernel32!lstrcmpiW
It would be much appreciated if anyone could provide some assistance in
solving this problem. Please let me know if you need more information.
Regards,
Simon
**********************************************************************
E-Mail Disclaimer - Please Read
Confidentiality:
This email and any files transmitted with it are confidential and intended solely for the individual to whom it is addressed. The information in this e-mail may not be used or disclosed except for the purpose for which it has been sent. If you are not the intended recipient you must not disclose it,
copy it, distribute it, or take any action based on its contents.
Please reply and highlight the error then delete it.
Security:
Please note that Internet email is not a fully secure method of
communication. You should be aware of this lack of security
when emailing us.
This email message has been swept by for the presence of computer viruses.
**********************************************************************
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx
" from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx
|