All, I forgot to mention that most of our traffic is over SSL. OpenSSL version is OpenSSL 1.0.1e-fips 11 Feb 2013. Thanks, -chris On 4/3/14, 10:04 AM, Christopher Schultz wrote: > All, > > I'm having a problem in production I've never seem before. We are > running a pair of AWS EC2 m1.micro web servers where only one of them in > really in service at any given time. The httpd instance serves some > static content and forwards a great deal of traffic via stunnel to a > single back-end Tomcat server using mod_jk 1.2.37. We have been running > under this configuration for several years with no problems. > > Last weekend, we upgraded our OS to Amazon Linux 2014.03 (32-bit) from > Amazon's previous version (I can't remember which one), including the > package-refresh that comes with it for httpd. The current kernel version > is 3.10.34. The current httpd version is 2.2.26. The package name is > "httpd-2.2.26-1.1.amzn1.i686" if anyone is interested. We are using a > prefork MPM with the following (default) settings: > > StartServers 8 > MinSpareServers 5 > MaxSpareServers 20 > ServerLimit 256 > MaxClients 256 > MaxRequestsPerChild 4000 > > What I can observe is that the CPU load average is rising from the usual > sub-2.0 value to sometimes as high as 70. That's seventy, not > seven-point-oh. > > I see no errors in the log, and httpd doesn't seem to be dropping any > requests... just running very very slowly. > > It seems to come in waves: the load will go up, and everything will slow > down, and then we'll get a reprieve. > > I can see 22 server processes running right this moment, but the load > average has dropped back to 0.05. > > I've enabled ExtendedStatus and it really doesn't look like there is a > huge number of requests being served. Less than 1 req/sec. This is *not* > a high-load server. I can see some of the httpd child processes using > 20% or more of the CPU as reported by 'top'. > > Is there a good way for me to determine what those processes are doing? > As this is a modestly-used server, I can probably enable additional > logging without too much trouble. > > Any help anyone can provide would be very much appreciated. > > Thanks, > -chris >
Attachment:
signature.asc
Description: OpenPGP digital signature
|