[users@httpd] Apache 2.0 hangs in apr_poll() when processing many requests

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi all,

We're having a problem with our Apache processes
getting stuck inside an apr_poll() call.  This is
happening on a production box and it only seems to
make itself a problem when the load gets up to > ~10
requests per second.

We're using Apache 2.0.46 (RedHat patch, prefork
version) in conjunction with PHP 4.3.2 (RedHat
patchlevel 23) on a dual-processor 3.6 GHz machine. 
We have 2 gig of memory and a 36Mb RAID array, running
under RedHat Linux AS3, kernel 2.4.21-32.ELsmp.

In general, processes seem to get hung at random
inside the apr_poll() call.  I'm no Apache expert, but
on a quick read of the stack trace, it looks like the
system is trying to wait for the socket write to
complete... except that it never does.

We're using PHP sessions, so when one server process
hangs, it leaves an exclusive lock open on the session
file.  If the user presses reload or attempts to
access another page, the following HTTPD processes
will block on accessing the session file.  This
quickly snowballs and puts us up past our MaxClients
setting, which shuts down the server.

I can't figure out how to reproduce the problem.  It's
taken out our server twice so far under heavy system
load, but I'm unable to make it happen at will.  Once,
I was able to repro the bug from my own browser.  On
the client's side, it submitted a page request for
"/", and it just hung there waiting for a response
(and got nothing).

If I wait during peak hours when lots of people are
using the machine, I can usually see a couple cases of
this from the server, but I've been strictly reactive
so far.  The PHP script and/or page being accessed
does not seem to matter.

We recently changed hardware and upgraded from Apache
1.3 to 2.0, so those could be potential factors. 
(However, we have two identical web servers that both
exhibit the same problem, so a hardware failure is
unlikely.)

We might have seen a similar problem (with lots of
HTTPD processes hanging around) under Apache 1.3 in
the old environment.  Unfortunately, our upgrade to
new hardware also coincides with a heavy increase in
traffic to our sites, so I can't necessarily point my
finger one way or another.

Can anyone make heads or trails out of the following
GDB stack trace?  I rebuilt libapr and httpd with
symbols so I could see what was going on, and I
managed to attach a debugger to the session and
capture the following.  Even if I let it run for a
while, it stays stuck inside the poll() call:

#0  0x00262689 in poll () from /lib/tls/libc.so.6
#1  0x00983c54 in apr_poll () from
/usr/lib/libapr-0.so.0
#2  0x009843a2 in apr_wait_for_io_or_timeout () from
/usr/lib/libapr-0.so.0
#3  0x00979d73 in apr_socket_sendv () from
/usr/lib/libapr-0.so.0
#4  0x0097a244 in apr_sendv () from
/usr/lib/libapr-0.so.0
#5  0x0807bb65 in writev_it_all (s=0x9a80e20,
vec=0xbfffa990, nvec=2, len=44839, nbytes=0xbfffa8e8)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/core.c:2864
#6  0x0807d038 in core_output_filter (f=0x9a812b0,
b=0x9cca288)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/core.c:4170
#7  0x0807483c in ap_pass_brigade (next=0x7531,
bb=0x1)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/util_filter.c:541
#8  0x08062d11 in ap_http_header_filter (f=0xa046638,
b=0x9a8bfb0)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/modules/http/http_protocol.c:1716
#9  0x0807483c in ap_pass_brigade (next=0x7531,
bb=0x1)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/util_filter.c:541
#10 0x08076edf in ap_content_length_filter
(f=0xa046620, b=0x9a8bfb0)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/protocol.c:1270
#11 0x0807483c in ap_pass_brigade (next=0x7531,
bb=0x1)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/util_filter.c:541
#12 0x08064aa4 in ap_byterange_filter (f=0xa046608,
bb=0x9a8bfb0)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/modules/http/http_protocol.c:3001
#13 0x0807483c in ap_pass_brigade (next=0x7531,
bb=0x1)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/util_filter.c:541
#14 0x0114fa2b in execute () from
/etc/httpd/modules/libphp4.so
#15 0x01122195 in php_ob_get_length () from
/etc/httpd/modules/libphp4.so
#16 0x0112222f in php_ob_get_length () from
/etc/httpd/modules/libphp4.so
#17 0x0112143f in php_end_ob_buffer () from
/etc/httpd/modules/libphp4.so
#18 0x01121790 in php_end_ob_buffers () from
/etc/httpd/modules/libphp4.so
#19 0x011108e8 in php_request_shutdown () from
/etc/httpd/modules/libphp4.so
#20 0x01150443 in execute () from
/etc/httpd/modules/libphp4.so
#21 0x011506a8 in execute () from
/etc/httpd/modules/libphp4.so
#22 0x0807483c in ap_pass_brigade (next=0x7531,
bb=0x1)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/util_filter.c:541
#23 0x0807c489 in default_handler (r=0x11f1818)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/core.c:3555
#24 0x080685c5 in ap_run_handler (r=0xa045960)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/config.c:200
#25 0x08068bdf in ap_invoke_handler (r=0xa045960)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/config.c:410
#26 0x08065266 in rename_original_env (p=0xa045960,
t=0xfffffffc)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/modules/http/http_request.c:317
#27 0x080608bc in ap_process_http_connection
(c=0x99450c8)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/modules/http/http_core.c:293
#28 0x08072195 in ap_run_process_connection
(c=0x9a80ef8)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/connection.c:85
#29 0x08066ae1 in child_main (child_num_arg=-4)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/mpm/prefork/prefork.c:706
#30 0x08066c34 in make_child (s=0x7531, slot=6)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/mpm/prefork/prefork.c:749
#31 0x08066d56 in perform_idle_server_maintenance
(p=0x8)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/mpm/prefork/prefork.c:841
#32 0x0806757d in ap_mpm_run (_pconf=0x97bf0a8,
plog=0x97e9150, s=0x97bf0a8)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/mpm/prefork/prefork.c:983
#33 0x0806da4f in main (argc=1, argv=0xbfffb614)
    at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/main.c:671

==

Also, in case it matters, here is what the scoreboard
looks like when the system gets wedged.  We have
plenty of memory and CPU cycles left when this
happens.


Apache Server Status for example.com

Server Version: Apache
Server Built: Feb 25 2005 09:35:45

Current Time: Thursday, 08-Sep-2005 15:01:42 CDT
Restart Time: Thursday, 08-Sep-2005 14:40:49 CDT
Parent Server Generation: 0
Server uptime: 20 minutes 53 seconds
Total accesses: 15303 - Total Traffic: 163.1 MB
CPU Usage: u869.65 s79.58 cu.29 cs.44 - 75.8% CPU load
12.2 requests/sec - 133.3 kB/second - 10.9 kB/request
150 requests currently being processed, 0 idle workers

WWWWWWWWWWWWWWWWWWWWWWCWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
WWWWWWWWWWWWWWWWWWWWWW..........................................
................................................................

Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R"
Reading Request,
"W" Sending Reply, "K" Keepalive (read), "D" DNS
Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully
finishing,
"I" Idle cleanup of worker, "." Open slot with no
current process

Srv	PID	Acc	M	CPU 	SS	Req	Conn	Child	Slot	Client	VHost
Request
0-0	14767	0/140/554	W 	9.24	60	0	0.0	1.40	5.57 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
1-0	14861	0/4/126	W 	0.83	19	0	0.0	0.08	2.33 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
2-0	14092	0/49/49	W 	18.10	1036	0	0.0	0.35	0.35 
65.162.244.194	example.com	GET
/projects/viewProject.php?_mode=search&_clearFlag=course,sp
3-0	14862	0/4/556	W 	1.03	20	0	0.0	0.10	4.59 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
4-0	14094	0/605/605	W 	71.52	55	0	0.0	7.80	7.80 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
5-0	14386	0/541/548	W 	36.02	46	0	0.0	5.89	5.90 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
6-0	14863	0/2/53	W 	0.63	30	0	0.0	0.05	0.75 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
7-0	14097	0/260/260	W 	26.50	48	0	0.0	2.20	2.20 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
8-0	14098	0/216/216	W 	37.65	6	0	0.0	4.00	4.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
9-0	14589	0/356/524	W 	39.67	55	0	0.0	3.79	5.60 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
10-0	14864	0/5/169	W 	0.73	45	0	0.0	0.05	1.83 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
11-0	14218	0/632/638	W 	58.11	0	0	0.0	7.82	7.88 
66.190.215.152	example.com	GET / HTTP/1.1
12-0	14102	0/47/47	W 	3.44	49	0	0.0	0.60	0.60 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
13-0	14865	0/3/138	W 	0.44	61	0	0.0	0.18	1.64 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
14-0	14866	0/5/249	W 	0.78	3	0	0.0	0.08	4.70 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
15-0	14105	0/608/608	W 	60.14	13	0	0.0	6.56	6.56 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
16-0	14280	0/583/587	W 	56.92	44	0	0.0	5.10	5.12 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
17-0	14867	0/3/22	W 	0.65	31	0	0.0	0.06	0.46 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
18-0	14868	0/5/105	W 	1.72	0	0	0.0	0.12	0.98 
209.223.94.229	example.com	GET /voodoo-status HTTP/1.0
19-0	14283	0/565/575	W 	51.57	52	0	0.0	5.97	6.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
20-0	14869	0/2/173	W 	0.54	8	0	0.0	0.05	1.60 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
21-0	14111	0/668/668	W 	49.32	36	0	0.0	5.97	5.97 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
22-0	14870	1/11/20	C 	1.89	0	0	11.4	0.16	0.33 
66.83.74.234	example.com	GET
/uploaded_files/deliverFile.php/hon%20icon.jpg
HTTP/1.1
23-0	14871	0/6/8	W 	1.02	43	0	0.0	0.06	0.07 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
24-0	14872	0/3/4	W 	0.35	53	0	0.0	0.03	0.04 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
25-0	14873	0/1/371	W 	0.29	56	0	0.0	0.03	4.66 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
26-0	14874	0/2/14	W 	0.58	5	0	0.0	0.05	0.10 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
27-0	14875	0/4/15	W 	1.06	0	0	0.0	0.07	0.17 
66.18.105.194	example.com	GET /r.php/1120902432
HTTP/1.1
28-0	14876	0/5/6	W 	0.53	53	0	0.0	0.25	0.25 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
29-0	14877	0/1/4	W 	0.31	46	0	0.0	0.03	0.13 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
30-0	14129	0/632/632	W 	58.34	59	0	0.0	6.87	6.87 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
31-0	14878	0/5/104	W 	0.63	43	0	0.0	0.05	1.35 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
32-0	14879	0/3/10	W 	0.42	47	0	0.0	0.04	0.06 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
33-0	14880	0/4/12	W 	0.92	39	0	0.0	0.08	0.14 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
34-0	14881	0/2/19	W 	0.65	42	0	0.0	0.05	0.13 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
35-0	14138	0/81/81	W 	5.14	56	0	0.0	1.04	1.04 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
36-0	14882	0/2/14	W 	0.42	49	0	0.0	0.04	0.30 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
37-0	14883	0/1/18	W 	0.30	50	0	0.0	0.03	0.07 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
38-0	14145	0/669/669	W 	50.81	47	0	0.0	6.89	6.89 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
39-0	14884	0/2/8	W 	0.40	45	0	0.0	0.18	0.22 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
40-0	14885	0/5/111	W 	0.77	37	0	0.0	0.05	1.21 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
41-0	14886	0/6/30	W 	0.54	50	0	0.0	0.06	0.27 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
42-0	14149	0/1/1	W 	0.00	1235	0	0.0	0.0000	0.0000 
64.132.138.66	example.com	POST /login/login.php
HTTP/1.1
43-0	14150	0/634/634	W 	56.34	57	0	0.0	5.24	5.24 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
44-0	14887	0/3/29	W 	0.61	41	0	0.0	0.05	0.23 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
45-0	14888	0/2/6	W 	0.51	39	0	0.0	0.04	0.10 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
46-0	14154	0/0/0	W 	0.00	1232	0	0.0	0.00	0.00 
64.132.138.66	example.com	POST /login/login.php
HTTP/1.1
47-0	14155	0/253/253	W 	14.58	38	0	0.0	2.02	2.02 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
48-0	14889	0/2/117	W 	0.43	51	0	0.0	0.04	0.95 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
49-0	14890	0/2/11	W 	0.34	52	0	0.0	0.03	0.06 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
50-0	14891	0/2/22	W 	0.34	47	0	0.0	0.03	0.12 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
51-0	14894	0/1/59	W 	0.32	35	0	0.0	0.03	0.81 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
52-0	14895	0/1/328	W 	0.30	33	0	0.0	0.03	2.40 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
53-0	14896	0/2/46	W 	0.32	32	0	0.0	0.03	0.54 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
54-0	14897	0/1/20	W 	0.29	28	0	0.0	0.03	0.22 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
55-0	14898	0/1/10	W 	0.30	26	0	0.0	0.03	0.10 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
56-0	14899	0/1/196	W 	0.33	27	0	0.0	0.03	2.12 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
57-0	14900	0/1/328	W 	0.27	30	0	0.0	0.03	2.98 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
58-0	14901	0/1/89	W 	0.28	21	0	0.0	0.03	0.85 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
59-0	14902	0/1/22	W 	0.35	18	0	0.0	0.03	0.10 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
60-0	14168	0/598/598	W 	54.77	54	0	0.0	7.50	7.50 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
61-0	14903	0/1/124	W 	0.33	24	0	0.0	0.03	1.94 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
62-0	14173	0/713/713	W 	59.91	38	0	0.0	7.55	7.55 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
63-0	14904	0/2/11	W 	0.43	17	0	0.0	0.05	0.07 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
64-0	14905	0/1/6	W 	0.33	22	0	0.0	0.03	0.05 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
65-0	14906	0/1/104	W 	0.28	24	0	0.0	0.03	0.85 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
66-0	14178	0/425/425	W 	41.45	51	0	0.0	4.58	4.58 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
67-0	14907	0/0/5	W 	0.42	71	0	0.0	0.00	0.05 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
68-0	14908	0/1/2	W 	0.32	25	0	0.0	0.03	0.05 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
69-0	14909	0/2/87	W 	0.37	10	0	0.0	0.03	0.95 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
70-0	14910	0/1/7	W 	0.35	16	0	0.0	0.03	0.04 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
71-0	14911	0/2/64	W 	0.61	5	0	0.0	0.04	0.91 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
72-0	14912	0/1/154	W 	0.28	14	0	0.0	0.03	1.10 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
73-0	14913	0/1/483	W 	0.33	12	0	0.0	0.03	4.37 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
74-0	14186	0/592/592	W 	44.76	9	0	0.0	5.73	5.73 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
75-0	14914	0/1/1	W 	0.31	11	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
76-0	14915	0/0/0	W 	0.00	70	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
77-0	14916	0/1/1	W 	0.35	10	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
78-0	14917	0/1/1	W 	0.28	9	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
79-0	14918	0/0/0	W 	0.00	70	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
80-0	14919	0/7/7	W 	0.72	42	0	0.0	0.07	0.07 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
81-0	14920	0/1/1	W 	0.29	7	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
82-0	14921	0/1/1	W 	0.27	7	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
83-0	14922	0/1/1	W 	0.33	8	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
84-0	14923	0/6/6	W 	1.06	1	0	0.0	0.11	0.11 
139.76.128.68	example.com	GET
/uploaded_files/deliverFile.php/playground%20hurricane.gif
85-0	14924	0/1/1	W 	0.34	6	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
86-0	14925	0/2/2	W 	0.33	13	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
87-0	14926	0/3/3	W 	0.43	31	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
88-0	14927	0/2/2	W 	0.31	14	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
89-0	14928	0/5/5	W 	0.48	29	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
90-0	14929	0/1/1	W 	0.32	29	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
91-0	14930	0/1/1	W 	0.32	11	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
92-0	14931	0/1/1	W 	0.27	28	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
93-0	14932	0/1/1	W 	0.32	26	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
94-0	14933	0/1/1	W 	0.31	27	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
95-0	14934	0/2/2	W 	0.33	16	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
96-0	14935	0/6/6	W 	0.49	34	0	0.0	0.07	0.07 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
97-0	14936	0/1/1	W 	0.27	19	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
98-0	14937	0/6/6	W 	0.56	43	0	0.0	0.04	0.04 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
99-0	14938	0/8/8	W 	0.44	41	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
100-0	14939	0/4/4	W 	0.10	67	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
101-0	14940	0/2/2	W 	0.28	17	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
102-0	14941	0/1/1	W 	0.31	25	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
103-0	14942	0/6/6	W 	0.70	37	0	0.0	0.06	0.06 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
104-0	14943	0/5/5	W 	0.44	34	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
105-0	14944	0/3/3	W 	0.34	15	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
106-0	14945	0/4/4	W 	0.53	32	0	0.0	0.05	0.05 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
107-0	14946	0/2/2	W 	0.28	22	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
108-0	14947	0/1/1	W 	0.29	15	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
109-0	14948	0/1/1	W 	0.30	23	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
110-0	14949	0/1/1	W 	0.28	21	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
111-0	14950	0/4/4	W 	0.39	35	0	0.0	0.04	0.04 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
112-0	14951	0/7/7	W 	0.71	36	0	0.0	0.05	0.05 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
113-0	14952	0/2/2	W 	0.36	18	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
114-0	14953	0/1/1	W 	0.35	20	0	0.0	0.03	0.03 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
115-0	14954	0/2/2	W 	0.43	33	0	0.0	0.04	0.04 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
116-0	14955	0/5/5	W 	0.44	39	0	0.0	0.05	0.05 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
117-0	14956	0/5/5	W 	0.67	4	0	0.0	0.04	0.04 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
118-0	14957	0/5/5	W 	0.09	65	0	0.0	0.01	0.01 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
119-0	14958	0/3/3	W 	0.64	2	0	0.0	0.05	0.05 
152.130.8.6	example.com	GET
/projects/viewProject.php?_mode=occurrenceView&_action=load
120-0	14959	0/1/1	W 	0.42	0	0	0.0	0.03	0.03 
65.87.105.1	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=browse&_action=f
121-0	14960	0/0/0	W 	0.00	64	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
122-0	14961	0/0/0	W 	0.00	64	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
123-0	14962	0/0/0	W 	0.00	64	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
124-0	14963	0/1/1	W 	0.00	64	0	0.0	0.0000	0.0000 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
125-0	14964	0/0/0	W 	0.00	63	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
126-0	14965	0/0/0	W 	0.00	63	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
127-0	14966	0/4/4	W 	0.34	62	0	0.0	0.20	0.20 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
128-0	14967	0/1/1	W 	0.01	63	0	0.0	0.0000	0.0000 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
129-0	14968	0/0/0	W 	0.00	63	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
130-0	14969	0/0/0	W 	0.00	63	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
131-0	14970	0/1/1	W 	0.00	63	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
132-0	14971	0/2/2	W 	0.28	62	0	0.0	0.02	0.02 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
133-0	14972	0/0/0	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
134-0	14973	0/1/1	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
135-0	14974	0/0/0	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
136-0	14975	0/0/0	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
137-0	14976	0/0/0	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
138-0	14977	0/0/0	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
139-0	14978	0/0/0	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
140-0	14979	0/0/0	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
141-0	14980	0/0/0	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
142-0	14981	0/1/1	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
143-0	14982	0/0/0	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
144-0	14983	0/0/0	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
145-0	14984	0/0/0	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
146-0	14985	0/0/0	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
147-0	14986	0/1/1	W 	0.08	61	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
148-0	14987	0/0/0	W 	0.00	62	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
149-0	14988	0/0/0	W 	0.00	61	0	0.0	0.00	0.00 
65.168.18.47	example.com	GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
Srv	Child Server number - generation
PID	OS process ID
Acc	Number of accesses this connection / this child /
this slot
M	Mode of operation
CPU	CPU usage, number of seconds
SS	Seconds since beginning of most recent request
Req	Milliseconds required to process most recent
request
Conn	Kilobytes transferred this connection
Child	Megabytes transferred this child
Slot	Total megabytes transferred this slot
Apache Server at www.example.com Port 80

Thanks!
SD



	
		
______________________________________________________
Click here to donate to the Hurricane Katrina relief effort.
http://store.yahoo.com/redcross-donate3/

---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@xxxxxxxxxxxxxxxx
   "   from the digest: users-digest-unsubscribe@xxxxxxxxxxxxxxxx
For additional commands, e-mail: users-help@xxxxxxxxxxxxxxxx



[Index of Archives]     [Open SSH Users]     [Linux ACPI]     [Linux Kernel]     [Linux Laptop]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Squid]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Video 4 Linux]     [Device Mapper]

  Powered by Linux