Re: Some pages loading very slow in 3.1.10 Stable

Eliezer <eliezer@xxxxxxxxxxxxxxxxxxxxx> · Mon, 24 Jan 2011 10:36:14 +0200

It's a small peace and most of the answers are not really suppose to there.

the first thing is that your cache is not just cache it's a store house..

it;'s not bad but you can try to change the ufs to aufs..

can get better performance.

what are the specification of the machine?

core i7 ?.... ram? what disk? array of disks?

did you tried to ping from the machine or WGET?

if it's debian you can install ifstat that can give you real-rime 
bandwidth usage and it might be cause of something else that is not 
related to squid..

if you have even like 25 clients downloading obsessively  mp3 files  for 
like 10 or more minutes in this time your I\O of your hard drive will rise..

also..

you can monitor the access to the squid folders onfly while you have the 
problem and to understand what is causing it..

if it's CPU load or DISK I\O load .. or other stuff.

by the way.. can you try the ubuntu squid3 stable?

im using squid3 stable on ubuntu 10.04 on an Intel Atom D450 machine 
with cache of 40GB and it's taking the load very nicely.

On 24/01/2011 09:12, Saiful Alam wrote:

Some results of TCPDUMP in -vv mode.

13:12:04.191180 IP (tos 0x0, ttl 127, id 2750, offset 0, flags [DF], proto TCP (6), length 40)
     172.16.80.2.1155>  77.67.29.42.www: Flags [.], cksum 0x6de4 (correct), seq 1127903567, ack 4192021369, win 64700, length 0
13:12:04.192822 IP (tos 0x0, ttl 64, id 4692, offset 0, flags [DF], proto TCP (6), length 823)
     www-12-02-snc5.facebook.com.www>  10.16.63.123.3714: Flags [P.], cksum 0xc789 (correct), seq 4235393899:4235394682, ack 3661185770, win 14600, length 783
13:12:04.831628 IP (tos 0x0, ttl 127, id 29487, offset 0, flags [DF], proto TCP (6), length 40)
     172.16.80.6.rmiregistry>  cds174.lax.llnw.net.www: Flags [.], cksum 0xfa45 (correct), seq 0, ack 460801, win 65535, length 0
13:12:05.355300 IP (tos 0x0, ttl 127, id 2265, offset 0, flags [DF], proto TCP (6), length 40)
     192.168.96.165.57362>  195.122.152.64.www: Flags [.], cksum 0xfc6c (correct), seq 1875916651, ack 4162436021, win 64240, length 0
13:12:05.355322 IP (tos 0x0, ttl 64, id 18785, offset 0, flags [DF], proto TCP (6), length 1500)
     195.122.152.64.www>  192.168.96.165.57362: Flags [.], cksum 0x0f8f (correct), seq 4381:5841, ack 0, win 6432, length 1460
13:12:05.355328 IP (tos 0x0, ttl 127, id 8325, offset 0, flags [DF], proto TCP (6), length 40)
     172.16.112.124.cvspserver>  8.27.4.126.www: Flags [.], cksum 0xfe6f (correct), seq 0, ack 662461, win 65535, length 0
13:12:05.655764 IP (tos 0x0, ttl 127, id 5393, offset 0, flags [DF], proto TCP (6), length 40)
     172.16.64.10.2308>  co63.reliablehosting.com.www: Flags [.], cksum 0x1b43 (correct), seq 63164814, ack 3610294926, win 65535, length 0
13:12:05.655789 IP (tos 0x0, ttl 64, id 65532, offset 0, flags [DF], proto TCP (6), length 1500)
     co63.reliablehosting.com.www>  172.16.64.10.2308: Flags [P.], cksum 0x4e5f (correct), seq 1461:2921, ack 0, win 17184, length 1460
13:12:05.655798 IP (tos 0x0, ttl 64, id 65533, offset 0, flags [DF], proto TCP (6), length 1215)
     co63.reliablehosting.com.www>  172.16.64.10.2308: Flags [P.], cksum 0xa23e (correct), seq 2921:4096, ack 0, win 17184, length 1175
13:12:05.656066 IP (tos 0x0, ttl 126, id 42247, offset 0, flags [DF], proto TCP (6), length 73)
     172.16.30.6.1575>  79-110-88-28.gfsrv.net.15103: Flags [P.], cksum 0x207b (correct), seq 22088:22121, ack 55870, win 64475, length 33
13:12:05.656081 IP (tos 0x0, ttl 126, id 42248, offset 0, flags [DF], proto TCP (6), length 51)
     172.16.30.6.1575>  79-110-88-28.gfsrv.net.15103: Flags [P.], cksum 0xd9a7 (correct), seq 22121:22132, ack 55870, win 64475, length 11
13:12:05.949960 IP (tos 0x0, ttl 126, id 5909, offset 0, flags [DF], proto TCP (6), length 40)
     172.16.10.9.1408>  GGCpmo1.pal.seabone.net.www: Flags [.], cksum 0xfe1a (correct), seq 3933402590, ack 133923280, win 65535, length 0
13:12:05.949983 IP (tos 0x0, ttl 126, id 31989, offset 0, flags [DF], proto TCP (6), length 52)
     172.16.52.6.62360>  c-n180-u0282-208.webazilla.com.www: Flags [.], cksum 0x0bbb (correct), seq 0, ack 1119485, win 64240, options [nop,nop,sack 1 {1120945:1129705}], length 0
13:12:05.949997 IP (tos 0x0, ttl 127, id 1202, offset 0, flags [DF], proto TCP (6), length 40)
     172.16.200.42.1454>  146.82.204.207.www: Flags [.], cksum 0x5a1a (correct), seq 0, ack 1965427, win 65375, length 0
13:12:05.950985 IP (tos 0x0, ttl 126, id 36156, offset 0, flags [DF], proto TCP (6), length 40)
     172.16.30.7.1795>  cds17.sin.llnw.net.www: Flags [.], cksum 0x779d (correct), seq 4079975438, ack 3825116412, win 65535, length 0
13:12:05.951132 IP (tos 0x0, ttl 126, id 5910, offset 0, flags [DF], proto TCP (6), length 996)
     172.16.10.9.1408>  GGCpmo1.pal.seabone.net.www: Flags [P.], cksum 0xf7ba (correct), seq 0:956, ack 1, win 65535, length 956
13:12:05.951152 IP (tos 0x0, ttl 64, id 20092, offset 0, flags [DF], proto TCP (6), length 40)
     GGCpmo1.pal.seabone.net.www>  172.16.10.9.1408: Flags [.], cksum 0xdc7e (correct), seq 1, ack 956, win 7648, length 0
13:12:05.955120 IP (tos 0x0, ttl 127, id 2284, offset 0, flags [DF], proto TCP (6), length 40)
     192.168.96.165.57362>  195.122.152.64.www: Flags [.], cksum 0xda34 (correct), seq 0, ack 8761, win 64240, length 0
13:12:05.955148 IP (tos 0x0, ttl 64, id 18790, offset 0, flags [DF], proto TCP (6), length 972)
     195.122.152.64.www>  192.168.96.165.57362: Flags [P.], cksum 0x02b4 (correct), seq 8761:9693, ack 0, win 6432, length 932
13:12:05.955605 IP (tos 0x0, ttl 126, id 20667, offset 0, flags [DF], proto TCP (6), length 40)
^C    172.16.106.3.49956>  213.254.248.146.www: Flags [.], cksum 0x696e (correct), seq 348423304, ack 3320774375, win 16425, length 0

226 packets captured
107740 packets received by filter
107433 packets dropped by kernel

Regards,
Saiful

----------------------------------------

From: saifulmr@xxxxxxxxxxx
To: rmkml@xxxxxxx
CC: squid-users@xxxxxxxxxxxxxxx
Date: Mon, 24 Jan 2011 18:06:02 +1100
Subject: RE:  Some pages loading very slow in 3.1.10 Stable

Thanks for your prompt reply. Well I am not much experienced admin in linux or in terms of squid, and therefore I havent installed wireshark/tshark/tcpdump in squid yet, but I will install it now to go in deep.

My previous version of squid was 2.7 Stable downloaded from aptitude which was installed in a different machine, and not in this new machine. My current configuration in the current PC (Core i7) is a squid 3.1.10 custom compiled by me with only the ZPH and the Linux Net Filter enabled. I haven't used any other parameters while configuring it.

Default Cache_Dir is splitted into three directories:
UFS /media/cache/storage501 51400 16 256
UFS /media/cache/storage502 51400 16 256
UFS /media/cache/storage503 51400 16 256

and cache_mem is 150MB

Sometimes during peak hours while the network load is high, we do experience very slow browsing (in all websites) which is usually resolved after a quick restart of the squid process/service.

Thanks&  Best Regards,
Shishir

----------------------------------------

Date: Mon, 24 Jan 2011 07:48:12 +0100
From: rmkml@xxxxxxx
To: saifulmr@xxxxxxxxxxx
CC: rmkml@xxxxxxx
Subject: Re:  Some pages loading very slow in 3.1.10 Stable

Hi Saiful,
On same plateform, do you have a old squid version working better? what is your previous squid version please?
can you start network capture like wireshark/tshark/tcpdump on your squid please? and run firebug for example and compare last and your previous squid version please?
can you compare squid cpu usage last and previous version please?
Regards
Rmkml

On Mon, 24 Jan 2011, Saiful Alam wrote:

Hello Everyone,
My Squid Configuration is pretty much default, except
the fact, that I have added some refresh_patterns myself and collected
from internet in order to get more hits. The server is a Squid3.1.10 (in
3128 intercept transparent) running on Ubuntu 10.10 and comprises of
Intel(R) Core(TM) i7 CPU 870 @ 2.93GHz. I would say that
everything is fine, except some pages are loading very very slow. For
example, www.music.com.bd / www.djmaza.com ... I will try to give more
examples as I find more, but for the time being these two sites are very
important for my clients. Don't understand actually whats wrong. If
this has anything to do with the squid.conf file, I will attach it with
the next reply. For the time being here's my refresh_patterns.

refresh_pattern ^http://.*\.facebook.com.*\.(jpg|gif|png|swf|wav|3gp|flv|swf|wmv|zip|rar) 10080 300% 40320 reload-into-ims
refresh_pattern ^http://.*\.fbcdn.net.*\.(jpg|gif|png|swf|wav|mp3|3gp|flv|swf|wmv|zip|rar) 10080 300% 40320 reload-into-ims
refresh_pattern ^http://.*\.yimg.com.*\.(jpg|gif|png|swf|wav|3gp|flv|swf|wmv|zip|rar) 10080 300% 40320 reload-into-ims
refresh_pattern ^http://.*\.ytimg.com.*\.(jpg|gif|png|swf|wav|mp3|3gp|flv|swf|wmv|zip|rar) 10080 300% 40320 reload-into-ims
refresh_pattern ^http://.*\.gstatic.com.*\.(jpg|gif|png) 10080 300% 40320 reload-into-ims
refresh_pattern ^http://.*\.google.com.*\.(jpg|gif|png) 10080 300% 40320 reload-into-ims
refresh_pattern dsebd\.org.*\.(jpg|gif|png) 600 50% 3000
refresh_pattern windowsupdate.com/.*\.(cab|exe) 4320 100% 43200 reload-into-ims
refresh_pattern download.microsoft.com/.*\.(cab|exe) 4320 100% 43200 reload-into-ims
refresh_pattern ^http://.*\.cnn\.com 1000 50% 4320 override-lastmod
refresh_pattern ^http://news\.bbc\.co\.uk 1000 50% 4320 override-lastmod
refresh_pattern microsoft 1080 150% 10080 override-lastmod
refresh_pattern msn\.com 4320 150% 10080 override-lastmod
refresh_pattern ^http://.*FIDO 360 1000% 480
refresh_pattern \.r[0-9][0-0]$ 10080 150% 40320
refresh_pattern ^http://.*\.asis$ 1440 50% 20160
refresh_pattern -i \.art$ 10080 150% 43200
refresh_pattern -i \.qtm$ 10080 150% 40320 reload-into-ims
refresh_pattern -i \.viv$ 10080 150% 40320 reload-into-ims
refresh_pattern -i \.pdf$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.jpg$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.png$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.gif$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.flv$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.ico$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.jpeg$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.x-flv$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.swf$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.wav$ 10080 300% 40320 reload-into-ims
refresh_pattern -i \.mp3$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.mp4$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.zip$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.rar$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.exe$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.zip$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.doc$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.ppt$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.tiff$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.ram$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.rm$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.rpm$ 10080 300% 40320 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.htm$ 1440 50% 40320 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.html$ 1440 50% 40320 override-expire override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.txt$ 1440 100% 20160 reload-into-ims override-lastmod
refresh_pattern -i \.arj$ 2880 200% 40320
refresh_pattern -i \.Z$ 10080 200% 40320
refresh_pattern ^ftp: 1440 50% 10080
refresh_pattern ^gopher: 1440 10% 1440
refresh_pattern . 0 40% 4320

negative_ttl 1 minute
positive_dns_ttl 15 hour
negative_dns_ttl 20 second

Thanks&  Best Regards,
Saiful