This issue has reared its ugly head once again for us. This time, the 'spyware' was Sun's Java autoupdater, which caused a single host to hit our proxy about 140 times per second most of yesterday, generating 1.3Gb of denials in our logs, which are normally from 300-600Mb per day. The server handled the load just fine, until it ran out of disk space trying to rotate the logs overnight. Here's a sample log entry: 1143608899.081 3 10.2.120.18 TCP_DENIED/407 945 GET http://java.sun.com/webapps/download/GetFile/1.5.0_03-b07/windows- i586/jre1.5.0_03.msi - NONE/- text/html I had a few replies in 2004 on how to deal with this problem, which I will re-visit, but I'm curious how others are dealing with this issue, and if any new ideas have come up since then. We are running 2.5stable9 on Mandrake 9.2. ------- Forwarded message follows ------- From: Shawn Wright <swright@xxxxxxxxx> To: squid-users@xxxxxxxxxxxxxxx Subject: More flexible logging options? Send reply to: swright@xxxxxxxxx Date sent: Tue, 23 Nov 2004 14:43:52 -0800 We are finding squid's logging options quite limited, and are wondering if there are any patches, or other ways to deal with some of the issues we encounter. For example, in the past few weeks, we've had numerous cases where a single client can generate 600Mb+ of log entries in a day, all caused by spyware hitting a small group of URLs many times per second. Of course, they are all denied, since we require authentication for all except a few cases, and the spyware doesn't pass credentials to the proxy. During times when our proxy is being assaulted by spyware, it spends a great deal of CPU time logging these denials. I would like to explore the possibility of one or more of the following: -handing off the logging to a separate process such as multilog -finding some way to place log limits where multiple lines from a single host would otherwise fill the logs. ie: maximum 5 denials logged per second per host, with a burst of 20. -limiting max # of connections allocated to a single IP per minute, since delay pools won't help when all the connections are denials (I don't think). Thanks for any suggestions. ------- End of forwarded message ------- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Shawn Wright, I.T. Manager Shawnigan Lake School http://www.sls.bc.ca swright@xxxxxxxxx
