|
>-----Original Message----- >From: RAY, DAVID [mailto:DAVID.RAY@xxxxxxxx]
>Sent: Tuesday, June 30, 2015 12:10 PM >To: users@xxxxxxxxxxxxxxxxx >Subject: AIX OS Patch Breaks Apache/Tomcat > >Hello. > >I am running Apache 2.2.29 and Tomcat 7.0.59 with tomcat connector(mod_jk) version 1.2.40 on AIX version 7.1 server. Started having problems this morning after AIX OS was patched to AIX 7.1 TL 03 SP 04 and
openssh to >6.0.0.6108. WebAdvisor runs fine immediately after apache is started or restarted. However its response slowing down. AIX server CPU steadily increases and approaches 100% after running for 5 or 10 minutes under heavy >load.. Seeing http process
accumulate. Not seeing much traffic at all in Tomcat server status. Recompiled apache, tomcat connector, and Tomcat native. Still no luck. Later determined there is a bug with openssh 6.0.0.6.108 >and tried 6.0.0.6109. Recompiled again. Still no improvement.
Just booted from Clone backup taken before the AIX patch and everything runs fine, no problems. Anyone else have this experience with AIX >patches/updates? Not really much to see in error logs. Tomcat is configured to run with java6 32 bit: > >IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 AIX ppc-32 jvmap3260sr16-20141216_227499 (JIT enabled, AOT enabled) >J9VM - 20141216_227499 >JIT - r9_20140523_64469ifx3 >GC - GA24_Java6_SR16_20141216_1020_B227499) >JCL - 20141216_01 > > >Thank you. >David We have determined that that this problem is triggered by Apache server-status. I have always used server-status to monitor our production system, no problem. I rarely use server-status to monitor our test
system because there is nothing to see there due to extremely light traffic. But after the AIX patch, server-status gradually causes the sever CPU to be maxed out. This problem was just duplicated on our test server. The CPU drops back to normal for that
server when server-status is stopped. Example of server-status URL I use in my IE 11 browser: https://website.com/server-status?refresh=5 Has anyone else had this experience? I guess for now, I will just have to stop using server-status to monitor Apache so that we can patch our AIX system. Interesting how server-status is actually showing the gradual CPU increase. When this snapshot was taken, server-status had just moved from line 9 to line 10, where the W is. The 53.55 at line 9 is what the
CPU usage was when this snapshot was taken: 0-0 17039448 0/13/13 _ 48.65 52 371 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/13/13 _ 49.14 47 305 0.0 0.10 0.10 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/13/13 _ 49.64 41 297 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/13/13 _ 50.46 35 554 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/13/13 _ 51.58 23 380 0.0 0.12 0.12 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/13/13 _ 51.04 29 373 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/13/13 _ 52.19 17 399 0.0 0.14 0.14 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/13/13 _ 52.93 11 528 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/13/13 _ 53.55 5 438 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/12/12 W 40.99 0 0 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/12/12 _ 43.98 104 256 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/12/12 _ 41.50 133 355 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/12/12 _ 42.01 127 338 0.0 0.10 0.10 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/11/11 _ 42.47 122 237 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/12/12 _ 42.93 116 325 0.0 0.10 0.10 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/12/12 _ 43.45 110 320 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/12/12 _ 44.41 99 264 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/12/12 _ 44.91 93 368 0.0 0.12 0.12 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/12/12 _ 46.99 70 378 0.0 0.08 0.08 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/12/12 _ 45.48 87 365 0.0 0.11 0.11 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/12/12 _ 45.91 82 256 0.0 0.10 0.10 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/12/12 _ 48.06 58 254 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/12/12 _ 46.41 76 294 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1
0-0 17039448 0/12/12 _ 47.53 64 368 0.0 0.09 0.09 xxx.xxx.xx.x website.com GET /server-status?refresh=5 HTTP/1.1 David |
|