Re: Sudden but sustained high bandwidth usage

[Heiler Bemerguy <heiler.bemerguy@xxxxxxxxxxxxxx>]

Hi guys

We're still getting all these SWAPFAIL and our link is skyrocketing...... please help! I think it didn't happen on older versions (.14 and below)

1457358929.643    953 10.23.0.63 TCP_SWAPFAIL_MISS/206 1450553 GET http://au.download.windowsupdate.com/c/msdownload/update/software/uprl/2013/10/wu-windows6.1-kb2882822-x64_60c6ad5c10b0be4e47a36b1e9e7f0ef807b1dda8.exe - HIER_DIRECT/201.30.251.43 application/octet-stream
1457358933.561   2157 10.23.0.63 TCP_SWAPFAIL_MISS/206 1932545 GET http://au.download.windowsupdate.com/c/msdownload/update/software/uprl/2013/10/wu-windows6.1-kb2882822-x64_60c6ad5c10b0be4e47a36b1e9e7f0ef807b1dda8.exe - HIER_DIRECT/201.30.251.43 application/octet-stream
1457358940.376    986 10.23.0.63 TCP_SWAPFAIL_MISS/206 976868 GET http://au.download.windowsupdate.com/c/msdownload/update/software/uprl/2013/11/wu-windows6.1-kb2888049-x64_ca3421cb93d78c14a94694692f63155c8639befc.exe - HIER_DIRECT/201.30.251.43 application/octet-stream
1457358960.737   8399 10.23.0.63 TCP_SWAPFAIL_MISS/206 1058138 GET http://au.download.windowsupdate.com/c/msdownload/update/software/uprl/2013/10/ie11-windows6.1-x64-en-us_ddec9ddc256ffa7d97831af148f6cc45130c6857.exe - HIER_DIRECT/201.30.251.43 application/octet-stream
1457358987.869  22464 10.88.10.5 TCP_SWAPFAIL_MISS/206 1416417 GET http://au.download.windowsupdate.com/d/msdownload/update/software/secu/2016/02/ie11-windows6.1-kb3141092-x64_0f7a98b9dc9f5c7ac73f6f543bf004a15e4d7be8.psf - HIER_DIRECT/201.30.251.26 application/octet-stream
1457359058.163  27916 10.88.10.5 TCP_SWAPFAIL_MISS/206 2629918 GET http://au.download.windowsupdate.com/d/msdownload/update/software/secu/2016/02/ie11-windows6.1-kb3141092-x64_0f7a98b9dc9f5c7ac73f6f543bf004a15e4d7be8.psf - HIER_DIRECT/201.30.251.40 application/octet-stream
1457359083.430  12771 10.12.0.186 TCP_SWAPFAIL_MISS/206 8680726 GET http://au.v4.download.windowsupdate.com/c/msdownload/update/software/secu/2015/10/mso-x-none_259d103bc1004338c277df19edf6a1c0b635d3cb.cab - HIER_DIRECT/131.253.33.50 application/octet-stream
1457359140.696  16267 10.88.10.5 TCP_SWAPFAIL_MISS_ABORTED/206 439 GET http://au.download.windowsupdate.com/d/msdownload/update/software/secu/2016/02/ie11-windows6.1-kb3141092-x64_0f7a98b9dc9f5c7ac73f6f543bf004a15e4d7be8.psf - HIER_DIRECT/201.30.251.40 application/octet-stream
1457359225.203  45105 10.88.10.5 TCP_SWAPFAIL_MISS/206 1084265 GET http://au.download.windowsupdate.com/d/msdownload/update/software/secu/2016/02/ie11-windows6.1-kb3134814-x64_63dd907558186498d92e71d95efaa89be8fb2dc7.psf - HIER_DIRECT/201.30.251.40 application/octet-stream
1457359301.824  31319 10.88.10.5 TCP_SWAPFAIL_MISS/206 579808 GET http://au.download.windowsupdate.com/d/msdownload/update/software/secu/2016/02/ie11-windows6.1-kb3134814-x64_63dd907558186498d92e71d95efaa89be8fb2dc7.psf - HIER_DIRECT/201.30.251.35 application/octet-stream
1457359348.878  14168 10.12.0.186 TCP_SWAPFAIL_MISS/206 13282361 GET http://au.v4.download.windowsupdate.com/d/msdownload/update/software/secu/2015/04/lync-x-none_a95899fce3a0b87d0e457233d689368405c304b7.cab - HIER_DIRECT/131.253.33.50 application/octet-stream
1457359525.229   7318 10.23.0.63 TCP_SWAPFAIL_MISS/206 71588 GET http://au.download.windowsupdate.com/c/msdownload/update/software/uprl/2013/10/ie11-windows6.1-x64-en-us_ddec9ddc256ffa7d97831af148f6cc45130c6857.exe - HIER_DIRECT/201.30.251.42 application/octet-stream
1457359528.617  24442 10.12.0.186 TCP_SWAPFAIL_MISS/206 3833367 GET http://au.v4.download.windowsupdate.com/d/msdownload/update/software/secu/2015/04/lync-x-none_a95899fce3a0b87d0e457233d689368405c304b7.cab - HIER_DIRECT/201.30.251.34 application/octet-stream
1457359567.653  40029 10.88.10.5 TCP_SWAPFAIL_MISS/206 919738 GET http://au.download.windowsupdate.com/d/msdownload/update/software/secu/2016/02/ie11-windows6.1-kb3141092-x64_0f7a98b9dc9f5c7ac73f6f543bf004a15e4d7be8.psf - HIER_DIRECT/201.30.251.27 application/octet-stream
1457359572.580  16448 10.12.0.186 TCP_SWAPFAIL_MISS/206 28526877 GET http://au.v4.download.windowsupdate.com/d/msdownload/update/software/secu/2015/04/mso-x-none_f2f3d0a088b32d1cb46eef0cb8b9d262fa733873.cab - HIER_DIRECT/201.30.251.34 application/octet-stream
1457359677.782  40737 10.88.10.5 TCP_SWAPFAIL_MISS/206 3439063 GET http://au.download.windowsupdate.com/d/msdownload/update/software/secu/2016/02/ie11-windows6.1-kb3141092-x64_0f7a98b9dc9f5c7ac73f6f543bf004a15e4d7be8.psf - HIER_DIRECT/201.30.251.40 application/octet-stream
1457359679.901  40323 10.12.0.186 TCP_SWAPFAIL_MISS/206 24791376 GET http://au.v4.download.windowsupdate.com/d/msdownload/update/software/secu/2015/04/mso-x-none_f2f3d0a088b32d1cb46eef0cb8b9d262fa733873.cab - HIER_DIRECT/201.30.251.26 application/octet-stream

Best Regards,

-- 
Heiler Bemerguy - (91) 98151-4894
Assessor Técnico - CINBESA (91) 3184-1751

Em 04/03/2016 18:54, Heiler Bemerguy escreveu:

Hi Amos,

It seems the "quick_abort_min -1 KB" did the trick. But I remember that "range_offset_limit" should overrule that.. isn't it?
Also, I saw people using -1 instead of "none" for range_offset_limit.. is it the same? :P

quick_abort_min -1 KB
acl wupdatecachable url_regex -i (microsoft|windowsupdate)\.com.*\.(cab|exe|ms[i|u|f]|dat|zip|psf|appx|appxbundle|esd)
range_offset_limit none wupdatecachable
refresh_pattern -i (microsoft|windowsupdate)\.com.*\.(cab|exe|ms[i|u|f]|dat|zip|psf|appx|appxbundle|esd) 483840 80% 483840 override-expire ignore-private ignore-no-store

Best Regards,

-- 
Heiler Bemerguy - (91) 98151-4894
Assessor Técnico - CINBESA (91) 3184-1751

Em 04/03/2016 01:01, Amos Jeffries escreveu:

On 4/03/2016 4:49 a.m., Heiler Bemerguy wrote:

Hi Amos,

You didn't notice it was always the same client ? The same IP address
redownloading ad eternum..

I managed to fix it by not caching stuff with "?" in it:

*refresh_pattern -i (/cgi-bin/|\?) 0 0% 0*

But I don't know if it's the best approach..

Provided you only added that refresh_pattern and not "cache deny" rules,
yes it is the best solution.
The refresh_pattern only applies to responses where there are missing
cacheability headers. So dynamic content which provides headers will
still be cached and served nicely.

The URL was like that:
/10.101.1.50 TCP_HIT/206 402 GET
//http://bg.v4.a.dl.ws.microsoft.com/dl/content/d/updt/2015/07/096c4bbc-4bc2-4ba1-8fd7-2e8cf3fb1937_132a7d6799d3bd625b0e5b375aa13552593bf0ed.appxbundle//?
- HIER_NONE/- application/octet-stream/

(After the "?" there were some variables)

Anyways, this isn't the cause of the ultra-high bandwidth load.. (*our
DL link is 100% used by squid right now!*). Most traffic comes from
windows updates...

/1457015568.658   9400 10.12.0.197 *TCP_SWAPFAIL_MISS/206* 1067290 GET
http://au.v4.download.*windowsupdate*.com/c/msdownload/update/software/crup/2015/02/publisher-x-none_08ccd79ac8a6bb475040360b6c9d8c9e1f258c9d.*cab
*- HIER_DIRECT/201.30.251.40 application/octet-stream//
//1457015624.067  36878 10.12.0.234 *TCP_MISS/206* 77842 GET
http://au.v4.download.*windowsupdate*.com/d/msdownload/update/software/crup/2014/02/windows8.1-kb2919355-x64_66955196a82751d1c8d9806d321487562b159f41.*psf
*- HIER_DIRECT/201.30.251.40 application/octet-stream//
//1457015750.556 126469 10.12.0.234 *TCP_MISS/206* 151183 GET
http://au.v4.download.*windowsupdate*.com/d/msdownload/update/software/crup/2014/02/windows8.1-kb2919355-x64_66955196a82751d1c8d9806d321487562b159f41.*psf
*- HIER_DIRECT/201.30.251.40 application/octet-stream//
//1457015753.263  11011 10.12.0.197 *TCP_MISS/206* 1616920 GET
http://au.v4.download.*windowsupdate*.com/c/msdownload/update/software/crup/2015/03/onenote-x-none_dd4f2bc75fc38be514c4009ce4d289e41f6b75d0.*cab
*- HIER_DIRECT/201.30.251.40 application/octet-stream//
//1457015780.978  13451 10.12.0.197 *TCP_SWAPFAIL_MISS/206* 2225824 GET
http://au.v4.download.*windowsupdate*.com/c/msdownload/update/software/crup/2015/03/onenote-x-none_dd4f2bc75fc38be514c4009ce4d289e41f6b75d0.*cab
*- HIER_DIRECT/201.30.251.40 application/octet-stream/

Do you see anything that could make it re-download over and over again
in this config?

The 206. If that is 206 from server Squid is unable to cache it for
future HITs.

/acl windowsupdate dstdomain .ws.microsoft.com
.windowsupdate.microsoft.com .update.microsoft.com .windowsupdate.com
.armdl.adobe.com//
//http_access allow windowsupdate//
//range_offset_limit none windowsupdate//
//

Can you try adding this:
  quick_abort_min -1 KB


Amos

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users