I'm looking for advice on figuring out what is causing intermittent high
CPU usage.
I'm seeing this on multiple servers - most of the time everything is
fine and I see the Squid workers using maybe 20% CPU each, but every so
often all the workers sit at the top of the process list in "top", using
> 97% CPU each and users report very sluggish web access.
Using squidclient during "sluggish" periods is also very slow, with
Squid taking several seconds to respond to the http requests. The
number of requests being handled by squid during the slow periods isn't
especially high (maybe ~20 / second) and is certainly lower than the
number of requests at other times - probably because it is taking so
long to answer requests, but this seems to indicate that it isn't simply
overloaded and having to deal with too many requests at once.
The during the "slow" periods, squid's servicing of requests seems very
bursty in nature - I see a whole bunch of requests over a few hundred
milliseconds and then nothing for maybe half a second. There are no log
entries that seem to coincide with these problems.
If I firewall off the clients, the load drops back to zero, so it seems
this is something a client is doing that is causing Squid to expend a
huge amount of CPU handling the request, rather than Squid getting stuck
in a loop or similar.
Restarting squid seems to temporarily fix the problem, but it invariably
comes back again at some point.
Notably the median service time go up:
HTTP Requests (All): 0.30178 0.40454
Cache Misses: 0.70906 0.65348
Cache Hits: 0.00000 0.00000
Near Hits: 0.00000 0.00000
Not-Modified Replies: 0.00000 0.00000
DNS Lookups: 0.02893 0.03092
ICP Queries: 0.00000 0.00000
UP Time: 11657.399 seconds
CPU Time: 8843.268 seconds
CPU Usage: 111.23%
CPU Usage, 5 minute avg: 144.81%
CPU Usage, 60 minute avg: 153.58%
Maximum Resident Size: 2937536 KB
Page faults with physical i/o: 3
Compared to (recently restarted):
HTTP Requests (All): 0.09477 0.09477
Cache Misses: 0.11465 0.11465
Cache Hits: 0.00000 0.00000
Near Hits: 0.00000 0.00000
Not-Modified Replies: 0.00000 0.00000
DNS Lookups: 0.00953 0.00953
ICP Queries: 0.00000 0.00000
UP Time: 293.336 seconds
CPU Time: 127.775 seconds
CPU Usage: 43.56%
CPU Usage, 5 minute avg: 47.40%
CPU Usage, 60 minute avg: 47.40%
Maximum Resident Size: 799808 KB
Page faults with physical i/o: 0
Is there any advice on how to track down what the problem is?
This Squid is doing:
- No caching
- ICAP
- External ACLs
- Auth (Negotiate and Basic)
- SSL bump
- Both TPROXY and non-transparent (majority of the traffic is
non-transparent)
- Uses an upstream proxy for most HTTP (not HTTPS)
--
- Steve Hill
Technical Director
Opendium Limited http://www.opendium.com
Direct contacts:
Instant messager: xmpp:steve@xxxxxxxxxxxx
Email: steve@xxxxxxxxxxxx
Phone: sip:steve@xxxxxxxxxxxx
Sales / enquiries contacts:
Email: sales@xxxxxxxxxxxx
Phone: +44-1792-824568 / sip:sales@xxxxxxxxxxxx
Support contacts:
Email: support@xxxxxxxxxxxx
Phone: +44-1792-825748 / sip:support@xxxxxxxxxxxx
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
