Search squid archive

Sudden but sustained high bandwidth usage

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




Hey guys.

For the third time, we got a sudden high bandwidth usage, almost saturating our link, and it won't stop until squid is restarted.
I'm totally SURE this inbound traffic comes from squid. It's like it's download stuff itself....



Look that after squid was restarted near 10:45, the network usage drops immediately and won't increase as high as before anymore..

This pattern started to happen when I changed from ROCK+AUFS to ROCK+ROCK, squid 3.5.14 x64.

Here's the most important conf settings.. I appreciate all comments about it.

acl windowsupdate dstdomain .ws.microsoft.com .windowsupdate.microsoft.com .update.microsoft.com .windowsupdate.com
http_access allow windowsupdate
range_offset_limit none windowsupdate
cache_mem 4 GB
maximum_object_size_in_memory 5 MB
memory_replacement_policy heap GDSF
cache_replacement_policy heap LFUDA
maximum_object_size 10 GB
cpu_affinity_map process_numbers=1,2,3,4,5,6 cores=1,2,3,4,5,6
workers 2
cache_dir rock /cache2/rock1 90000 min-size=0 max-size=32768
cache_dir rock /cache/rock1 300000 min-size=32768 max-size=10737418240
store_dir_select_algorithm round-robin
read_ahead_gap 4096 KB
client_request_buffer_max_size 2048 KB
dns_v4_first on
ipcache_size 80000
fqdncache_size 40000
memory_pools on
memory_pools_limit 150 MB
reload_into_ims on
connect_retries 3
cache_swap_low 98
cache_swap_high 99
store_avg_object_size 92 KB
client_idle_pconn_timeout 30 seconds
client_persistent_connections off
server_persistent_connections off

error.log right in this moment:
08:55:29 kid1| local=10.1.10.9:3080 remote=10.107.0.71:54515 FD 3665 flags=1: read/write failure: (32) Broken pipe
09:00:02 kid2| snmpHandleUdp: FD 55 recvfrom: (11) Resource temporarily unavailable
09:00:02 kid1| snmpHandleUdp: FD 29 recvfrom: (11) Resource temporarily unavailable
09:02:14 kid2| WARNING: Closing client connection due to lifetime timeout
09:02:14 kid2|       http://prod.video.msn.com/tenant/amp/entityid/BBq7uZY?blobrefkey=103&$blob=1
09:03:34 kid1| WARNING: HTTP: Invalid Response: Bad header encountered from http://sable.madmimi.com/view?id=24371.4971993.01561ff3
e8e7c09ac362ded25f80a76b AKA sable.madmimi.com/view?id=24371.4971993.01561ff3e8e7c09ac362ded25f80a76b
09:03:38 kid1| WARNING: Closing client connection due to lifetime timeout
09:03:38 kid1|       http://download.windowsupdate.com/d/msdownload/update/software/defu/2016/02/am_delta_patch_1.213.7305.0_59c57a
caccbdfa7fa9dd5574f0a7ded60de11963.exe
09:04:15 kid1| WARNING: HTTP: Invalid Response: Bad header encountered from http://sable.madmimi.com/view?id=24371.4931972.b5133065
c861d91790f59bf39ef1abf3 AKA sable.madmimi.com/view?id=24371.4931972.b5133065c861d91790f59bf39ef1abf3
09:04:28 kid2| WARNING: Closing client connection due to lifetime timeout
09:04:28 kid2|       http://www.ingressocerto.com/facet-search.json?f=/p-data-Offset:2
09:04:42 kid2| Could not parse headers from on disk object
09:05:02 kid1| snmpHandleUdp: FD 29 recvfrom: (11) Resource temporarily unavailable
09:05:02 kid2| snmpHandleUdp: FD 55 recvfrom: (11) Resource temporarily unavailable
09:05:02 kid1| snmpHandleUdp: FD 29 recvfrom: (11) Resource temporarily unavailable
09:05:18 kid2| SECURITY ALERT: Missing hostname in URL 'http://'. see access.log for details.
09:06:32 kid2| SECURITY ALERT: Missing hostname in URL 'http://'. see access.log for details.
09:06:32 kid2| SECURITY ALERT: Missing hostname in URL 'http://'. see access.log for details.
09:06:36 kid2| clientIfRangeMatch: Weak ETags are not allowed in If-Range: "141811-1456747291000" ? "141811-1456747291000"
09:08:14 kid2| urlParse: URL too large (8360 bytes)
09:09:40 kid2| SECURITY ALERT: Missing hostname in URL 'http://'. see access.log for details.
09:10:03 kid1| clientIfRangeMatch: Weak ETags are not allowed in If-Range: "94779-1455643401000" ? "94779-1455643401000"
09:14:01 kid1| local=10.1.10.9:3080 remote=10.101.2.60:1350 FD 2713 flags=1: read/write failure: (32) Broken pipe
09:14:14 kid2| Could not parse headers from on disk object
09:14:18 kid2| SECURITY ALERT: Missing hostname in URL 'http://'. see access.log for details.
09:15:25 kid2| WARNING: Closing client connection due to lifetime timeout
09:15:25 kid2|       http://prod.video.msn.com/tenant/amp/entityid/BBq8oCY?blobrefkey=103&$blob=1
09:18:01 kid2| Could not parse headers from on disk object
09:19:42 kid1| clientIfRangeMatch: Weak ETags are not allowed in If-Range: "10819-1456747291000" ? "10819-1456747291000"
09:20:03 kid2| snmpHandleUdp: FD 55 recvfrom: (11) Resource temporarily unavailable
09:20:27 kid2| WARNING: HTTP: Invalid Response: Bad header encountered from http://sable.madmimi.com/view?id=24371.4894720.98d08b18
9bb8855e833cecd4ea9e7e7e AKA sable.madmimi.com/view?id=24371.4894720.98d08b189bb8855e833cecd4ea9e7e7e
09:21:06 kid2| local=10.1.10.9:3080 remote=10.42.0.43:50042 FD 1878 flags=1: read/write failure: (110) Connection timed out
09:21:23 kid2| WARNING: HTTP: Invalid Response: Bad header encountered from http://sable.madmimi.com/view?id=24371.4870485.934d4afb
07878189bf41826766b1db26 AKA sable.madmimi.com/view?id=24371.4870485.934d4afb07878189bf41826766b1db26
09:23:04 kid1| local=10.1.10.9:3080 remote=10.102.2.19:49536 FD 1770 flags=1: read/write failure: (110) Connection timed out
09:23:33 kid1| urlParse: URL too large (8268 bytes)
09:23:56 kid2| local=10.1.10.9:4816 remote=216.58.222.35:443 FD 1046 flags=1: read/write failure: (32) Broken pipe
09:25:02 kid1| snmpHandleUdp: FD 29 recvfrom: (11) Resource temporarily unavailable
09:25:32 kid1| local=10.1.10.9:3080 remote=10.12.0.138:49951 FD 3338 flags=1: read/write failure: (32) Broken pipe
09:25:49 kid1| urlParse: URL too large (8231 bytes)
09:27:24 kid1| urlParse: URL too large (8231 bytes)
09:27:46 kid2| urlParse: URL too large (10742 bytes)
09:28:16 kid1| local=10.1.10.9:36318 remote=216.58.222.35:443 FD 2169 flags=1: read/write failure: (32) Broken pipe
09:28:58 kid1| Could not parse headers from on disk object
09:29:11 kid1| local=10.1.10.9:44504 remote=216.58.222.46:443 FD 4783 flags=1: read/write failure: (32) Broken pipe
09:29:20 kid1| Could not parse headers from on disk object
09:30:03 kid1| snmpHandleUdp: FD 29 recvfrom: (11) Resource temporarily unavailable
09:31:13 kid1| local=10.1.10.9:3080 remote=10.12.0.147:49815 FD 4230 flags=1: read/write failure: (32) Broken pipe
09:31:25 kid2| SECURITY ALERT: Missing hostname in URL 'http://'. see access.log for details.
09:31:26 kid1| SECURITY ALERT: Missing hostname in URL 'http://'. see access.log for details.
09:31:46 kid1| Starting new redirector helpers...
09:31:46 kid1| helperOpenServers: Starting 20/70 'ufdbgclient' processes
09:31:50 kid2| local=10.1.10.9:63724 remote=74.207.230.168:443 FD 2461 flags=1: read/write failure: (32) Broken pipe
09:33:51 kid2| local=10.1.10.9:3080 remote=10.107.0.71:56817 FD 1766 flags=1: read/write failure: (32) Broken pipe
09:35:15 kid2| Could not parse headers from on disk object
09:40:03 kid1| snmpHandleUdp: FD 29 recvfrom: (11) Resource temporarily unavailable
09:42:03 kid2| clientIfRangeMatch: Weak ETags are not allowed in If-Range: "968-1455828160000" ? "968-1455828160000"
09:43:33 kid1| local=10.1.10.9:59631 remote=216.58.222.34:443 FD 2720 flags=1: read/write failure: (32) Broken pipe
09:45:32 kid2| Could not parse headers from on disk object
09:46:30 kid2| local=10.1.10.9:3080 remote=10.23.0.44:49826 FD 2704 flags=1: read/write failure: (110) Connection timed out
09:46:30 kid1| local=10.1.10.9:3080 remote=10.101.4.25:58434 FD 4631 flags=1: read/write failure: (110) Connection timed out
09:46:55 kid1| local=10.1.10.9:3080 remote=10.11.0.186:63418 FD 159 flags=1: read/write failure: (110) Connection timed out
09:47:33 kid1| clientIfRangeMatch: Weak ETags are not allowed in If-Range: "a80649-16652-52ce81a00d4dd" ? "a80649-16652-52ce81a00d4
dd"
09:50:03 kid1| snmpHandleUdp: FD 29 recvfrom: (11) Resource temporarily unavailable
09:50:28 kid1| Could not parse headers from on disk object
09:50:28 kid1| varyEvaluateMatch: Oops. Not a Vary object on second attempt, 'http://pix04.revsci.net/D08734/a1/0/3/0.js?DM_LOC=%3D
http%3A%2F%2Fna.com%3FdlxInitiated%3Dtrue%26nada%3D%26naid%3D2015121611542932923036123812%26namp%3D' 'accept-encoding="gzip,%20deflate,%20sdch
"'
09:50:28 kid1| clientProcessHit: Vary object loop!
09:50:46 kid1| helperHandleRead: unexpected reply on channel 0 from redirector #Hlpr301 'OK'
09:50:46 kid1| helperHandleRead: unexpected reply on channel 0 from redirector #Hlpr301 'OK'
09:52:22 kid1| local=10.1.10.9:3080 remote=10.88.11.1:49932 FD 75 flags=1: read/write failure: (110) Connection timed out
09:53:14 kid2| local=10.1.10.9:3080 remote=10.1.4.168:9637 FD 3038 flags=1: read/write failure: (110) Connection timed out
09:54:07 kid1| Could not parse headers from on disk object
09:54:10 kid1| Could not parse headers from on disk object
09:54:12 kid2| urlParse: URL too large (8300 bytes)
09:54:17 kid2| Could not parse headers from on disk object
09:55:02 kid1| snmpHandleUdp: FD 29 recvfrom: (11) Resource temporarily unavailable
09:59:25 kid1| local=10.1.10.9:3080 remote=10.61.0.128:3955 FD 1589 flags=1: read/write failure: (32) Broken pipe
10:01:37 kid2| local=10.1.10.9:3080 remote=10.11.0.125:60064 FD 1663 flags=1: read/write failure: (110) Connection timed out
10:03:06 kid2| WARNING: Closing client connection due to lifetime timeout
10:03:06 kid2|       http://r1---sn-p5qlsu7s.gvt1.com/edgedl/release2/u0bdrsci12g6umtr3oo309iyvgxltyrngktaw787s86k670suu6f9v28nnnnj
wjgjpd8vbs8iw7mpw7vh9r3ad5al7r607gwgwz/48.0.2564.116_48.0.2564.109_chrome_updater.exe?expire=1456764484&ip=177.74.57.130&ipbits=0&pl=24&shardb
ypass=yes&sparams=expire,ip,ipbits,mm,mn,ms,mv,nh,pl,shardbypass&signature=7BD8296B68294B5A2BF5A09E26F963E5E721CA22.7E237A019B036E4D5A5C9CDA30
CCF0CBA2932C45&key=cms1&req_id=81d1131ec532cde1&redirect_counter=4&cms_redirect=yes&mm=26&mn=sn-p5qlsu7s&ms=tsu&mt=1456750037&mv=m
10:03:19 kid2| Could not parse headers from on disk object
10:03:22 kid2| Could not parse headers from on disk object
10:03:30 kid2| Could not parse headers from on disk object
10:03:38 kid1| Could not parse headers from on disk object
10:03:46 kid2| Could not parse headers from on disk object
10:05:03 kid2| snmpHandleUdp: FD 55 recvfrom: (11) Resource temporarily unavailable
10:05:28 kid2| local=10.1.10.9:3080 remote=10.122.0.114:50243 FD 5349 flags=1: read/write failure: (32) Broken pipe
10:10:02 kid1| snmpHandleUdp: FD 29 recvfrom: (11) Resource temporarily unavailable
10:10:13 kid1| urlParse: URL too large (8335 bytes)
10:11:01 kid1| local=10.1.10.9:3080 remote=10.1.4.17:50039 FD 1621 flags=1: read/write failure: (110) Connection timed out
10:11:07 kid1| local=10.1.10.9:3080 remote=10.1.3.247:56870 FD 322 flags=1: read/write failure: (32) Broken pipe

Best Regards,

-- 
Heiler Bemerguy - (91) 98151-4894
Assessor Técnico - CINBESA (91) 3184-1751
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux