On 16/04/2015 3:16 p.m., iridium191 wrote: > Hi Forum, > > We're running Squid Version 3.3.8 (from repository) on Ubuntu server 14.04 > LTS x64 as a caching proxy server for a mixed Linux/Windows network. > We were attempting to cache apt-get and Windows updates using various > refresh patterns but commented them out a few weeks ago after an issue where > the squid server had downloaded over 100GB of data over 24 hours but in that > time had served only about 6GB to clients. 100GB represents almost our > entire normal monthly traffic. Most of the incoming Squid traffic was from > Windows updates or Akamai servers. Prior to that the server had been running > for months without an issue. > Unfortunately the same thing happened again last night. The only remanent of > the Windows Update cache commands we forgot to uncomment in squid.conf were: > range_offset_limit 200 MB WindowsUpdate > maximum_object_size 1 GB > quick_abort_min -1 > > which we have since commented out and restarted the Squid server. Leave the maximum_object_size, the default is to only cache 4 MB objects and anything else becomes a MISS to consume more upstream bandwidth. The range_offset_limit and quick_abort_min are what causes more download for Range requests than what clients requested. > > The incoming traffic to Squid is being recorded by the syslog of our ASA5510 > firewall and corroborated by our ISP's management console. Clients can only > access the internet through the proxy. The outgoing traffic statistics are > being reported by SARG, SquidAnalyzer and a custom tcpdump script of all > traffic from the server and all three agree. > > So our question- is there any way that Squid can download content > repeatedly, and many times in excess of what our clients are actually > requesting? How can we debug this issue further? Several. * Remote server not supporting HTTP/1.1 304 revalidation properly. Results in the server delivering 200 response + full object data to Squid when a 304 would suffice and Squid only delivers 304 to the client. * Misconfigured / open proxy. When remote users abuse the proxy all traffic goes over the WAN interfaces. Measurement tools that differentiate by NIC on the Squid machine will see this as massive amounts of WAN traffic relative to LAN. * Long polling connections. * ICAP. The ICAP service may be doing anything that triggers lots of WAN traffic use. It can require full objects - forcing the same behaviour as quick_abort_*, it can do its own downloads - traffic not used by Squid at all but still used, it can make lots of requests * squidguard. Same as with ICAP it can be doing its own downloads. Also URL-rewritten traffic does not relay the client reuqest to the server, but the re-writer generated request. This is another way server 200 responses can become 200 when only 304 were necessary. > relevant squid.conf Is missing many configuration details that must be present for an operational proxy. Specifically the access controls to determine if the open proxy issues are relevant. With all that logging and traffic measurement are you able to track down an example transaction where Squid fetches more from the network than it delivers to the client. Squids cachemgr utilization report has a breakdown of how much traffic is used by each I/O type. It can help identify where to look at closer. Amos _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
