Hi All,
I need a help on this issue. On heavy network traffic with squid
running, link bandwidth is not utilized properly. If I bypass squid,
my link bandwidth is utilized properly.
Updated topology:
=============
(10 Mbps Link)
client< ------- > Squid Box <-------> Proxy client<------> Proxy
server<---> webserver
During problem scenario, I could see more tcp sessions with FIN_WAIT_1
state in Proxy server . I also observed that Recv -q in CLOSE_WAIT
state is getting increased in Squid Box. The number of tcp sessions
from Squid to webserver are also getting dropped drastically.
Squid.conf
========
http_port 3128 tproxy transparent
http_port 80 accel defaultsite=xyz.abc.com
hierarchy_stoplist cgi-bin
acl VIDEO url_regex ^http://fa\.video\.abc\.com
cache allow VIDEO
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
cache_mem 100 MB
cache_swap_low 70
cache_swap_high 80
maximum_object_size 51200 KB
maximum_object_size_in_memory 10 KB
ipcache_size 8192
fqdncache_size 8192
cache_replacement_policy heap LFUDA
memory_replacement_policy heap LFUDA
cache_dir aufs //var/logs/cache 6144 16 256
access_log //var/logs/access.log squid
cache_log //var/logs/cache.log
cache_store_log none
mime_table //var/opt/abs/config/acpu/mime.conf
pid_filename //var/run/squid.pid
refresh_pattern -i fa.video.abc.com/* 600 0% 600 override-expire
override-lastmod reload-into-ims ignore-reload
refresh_pattern -i video.abc.com/* 600 0% 600 override-expire
override-lastmod reload-into-ims ignore-reload
refresh_pattern -i media.abc.com/* 600 0% 600 override-expire
override-lastmod reload-into-ims ignore-reload
refresh_pattern -i xyz.abc.com/.*\.js 600 200% 600 override-expire
override-lastmod reload-into-ims
refresh_pattern -i xyz.abc.com/.*\.gif 600 200% 600 override-expire
override-lastmod reload-into-ims
refresh_pattern -i xyz.abc.com/.*\.jpg 600 200% 600 override-expire
override-lastmod reload-into-ims
refresh_pattern -i xyz.abc.com/.*\.jpg 600 200% 600 override-expire
override-lastmod reload-into-ims
refresh_pattern -i xyz.abc.com/.*\.png 600 200% 600 override-expire
override-lastmod reload-into-ims
refresh_pattern -i xyz.abc.com/.*\.css 600 200% 600 override-expire
override-lastmod reload-into-ims
refresh_pattern -i ^http://.wsj./.* 10 200% 10 override-expire
override-lastmod reload-into-ims ignore-reload
refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 480 100% 480
override-expire override-lastmod reload-into-ims
refresh_pattern -i \.(htm|html|js|css)$ 480 100% 480 override-expire
override-lastmod reload-into-ims
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
negative_ttl 1 minutes
positive_dns_ttl 1800 seconds
forward_timeout 2 minutes
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 21
acl Safe_ports port 443
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 1025-65535
acl Safe_ports port 280
acl Safe_ports port 488
acl Safe_ports port 591
acl Safe_ports port 777
acl CONNECT method CONNECT
acl video_server dstdomain cs.video.abc.com
always_direct allow video_server
acl PURGE method PURGE
http_access allow PURGE localhost
http_access deny PURGE
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT all
http_access allow all
icp_access allow all
tcp_outgoing_address 172.19.134.2
visible_hostname 172.19.134.2
server_persistent_connections off
logfile_rotate 1
error_map http://localhost:1000/abp/squidError.do 404
memory_pools off
store_objects_per_bucket 100
strip_query_terms off
coredump_dir //var/cache
store_dir_select_algorithm round-robin
cache_peer 172.19.134.2 parent 1000 0 no-query no-digest originserver
name=aportal
cache_peer www.abc.com parent 80 0 no-query no-digest originserver name=dotcom
cache_peer guides.abc.com parent 80 0 no-query no-digest originserver
name=travelguide
cache_peer selfcare.abc.com parent 80 0 no-query no-digest
originserver name=selfcare
cache_peer abcd.mediaroom.com parent 80 0 no-query no-digest
originserver name=mediaroom
acl webtrends url_regex ^http://statse\.webtrendslive\.com
acl the_host dstdom_regex xyz\.abc\.com
acl abp_regex url_regex ^http://xyz\.abc\.com/abp
acl gbp_regex url_regex ^http://xyz\.abc\.com/gbp
acl abcdstatic_regex url_regex ^http://xyz\.goginflight\.com/static
acl dotcom_regex url_regex ^www\.abc\.com
acl dotcomstatic_regex url_regex ^www\.abc\.com/static
acl travelguide_regex url_regex ^http://guides\.abc\.com
acl selfcare_regex url_regex ^http://selfcare\.abc\.com
acl mediaroom_regex url_regex ^http://abcd\.mediaroom\.com
never_direct allow abp_regex
cache_peer_access aportal allow abp_regex
acl PURGE method PURGE
http_access allow PURGE localhost
http_access deny PURGE
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT all
http_access allow all
icp_access allow all
tcp_outgoing_address 172.19.134.2
visible_hostname 172.19.134.2
server_persistent_connections off
logfile_rotate 1
error_map http://localhost:1000/abp/squidError.do 404
memory_pools off
store_objects_per_bucket 100
strip_query_terms off
coredump_dir //var/cache
store_dir_select_algorithm round-robin
cache_peer 172.19.134.2 parent 1000 0 no-query no-digest originserver
name=aportal
cache_peer www.abc.com parent 80 0 no-query no-digest originserver name=dotcom
cache_peer guides.abc.com parent 80 0 no-query no-digest originserver
name=travelguide
cache_peer selfcare.abc.com parent 80 0 no-query no-digest
originserver name=selfcare
cache_peer abcd.mediaroom.com parent 80 0 no-query no-digest
originserver name=mediaroom
acl webtrends url_regex ^http://statse\.webtrendslive\.com
acl the_host dstdom_regex xyz\.abc\.com
acl abp_regex url_regex ^http://xyz\.abc\.com/abp
acl gbp_regex url_regex ^http://xyz\.abc\.com/gbp
acl abcdstatic_regex url_regex ^http://xyz\.goginflight\.com/static
acl dotcom_regex url_regex ^www\.abc\.com
acl dotcomstatic_regex url_regex ^www\.abc\.com/static
acl travelguide_regex url_regex ^http://guides\.abc\.com
acl selfcare_regex url_regex ^http://selfcare\.abc\.com
acl mediaroom_regex url_regex ^http://abcd\.mediaroom\.com
never_direct allow abp_regex
cache_peer_access aportal allow abp_regex
acl PURGE method PURGE
http_access allow PURGE localhost
http_access deny PURGE
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT all
http_access allow all
icp_access allow all
tcp_outgoing_address 172.19.134.2
visible_hostname 172.19.134.2
server_persistent_connections off
logfile_rotate 1
error_map http://localhost:1000/abp/squidError.do 404
memory_pools off
store_objects_per_bucket 100
strip_query_terms off
coredump_dir //var/cache
store_dir_select_algorithm round-robin
cache_peer 172.19.134.2 parent 1000 0 no-query no-digest originserver
name=aportal
cache_peer www.abc.com parent 80 0 no-query no-digest originserver name=dotcom
cache_peer guides.abc.com parent 80 0 no-query no-digest originserver
name=travelguide
cache_peer selfcare.abc.com parent 80 0 no-query no-digest
originserver name=selfcare
cache_peer abcd.mediaroom.com parent 80 0 no-query no-digest
originserver name=mediaroom
acl webtrends url_regex ^http://statse\.webtrendslive\.com
acl the_host dstdom_regex xyz\.abc\.com
acl abp_regex url_regex ^http://xyz\.abc\.com/abp
acl gbp_regex url_regex ^http://xyz\.abc\.com/gbp
acl abcdstatic_regex url_regex ^http://xyz\.goginflight\.com/static
acl dotcom_regex url_regex ^www\.abc\.com
acl dotcomstatic_regex url_regex ^www\.abc\.com/static
acl travelguide_regex url_regex ^http://guides\.abc\.com
acl selfcare_regex url_regex ^http://selfcare\.abc\.com
acl mediaroom_regex url_regex ^http://abcd\.mediaroom\.com
never_direct allow abp_regex
cache_peer_access aportal allow abp_regex
cache_peer_access aportal allow abp_regex
cache_peer_access dotcom allow dotcom_regex
cache_peer_access dotcom allow dotcomstatic_regex
cache_peer_access travelguide allow travelguide_regex
cache_peer_access selfcare allow selfcare_regex
cache_peer_access mediaroom allow mediaroom_regex
cache deny webtrends
Do i need to tune squid.conf / tcp parameters in order to address this issue?
Please share your suggestions on this.
Regards,
Saravanan N
On Tue, Nov 26, 2013 at 5:54 PM, SaRaVanAn
<saravanan.nagarajan87@xxxxxxxxx> wrote:
> On Tue, Nov 26, 2013 at 5:16 PM, Antony Stone
> <Antony.Stone@xxxxxxxxxxxxxxxxxxxx> wrote:
>> On Tuesday 26 November 2013 at 11:37, SaRaVanAn wrote:
>>
>>> Hi All,
>>> I am doing a small test for bandwidth measurement of my test setup
>>> while squid is running. I am running a script to pump the traffic from
>>> client browser to Web-server via Squid box.
>>
>> Er, do you really mean you are sending data from the browser to the server?
>>
>>> The script creates around 50 user sessions and tries to do wget of randomly
>>> selected dynamic URL's.
>>
>> That sounds more standard - wget will fetch data from the server to the
>> browser.
> =========
> The script randomly picks the URL from the list of URL's
> defined in a file and tries to fetch that URL.
>
>>
>> What do you mean by "dynamic URLs"? Where / how is the content actually being
>> generated?
>>
> ==========
> Its a standard list of URL's with question mark in the
> end to avoid Squid caching.
> For example : www.espncricinfo.com?
>
>>> After some time,
>>
>> Please define.
>>
> ==========
> After 15-20 minutes from the time of execution of script.
>
>>> I'm observing a drop in bandwidth of the link,
>>
>> Please define - what network setup are you using - what bandwidth are you
>> getting at the start. what level does it drop to, does it return to the
>> previous level?
>>
> ============
> eth0 eth1
> Windows Laptop --------- Linux machine(Squid Running) ----- Internet
>
> We are measuring the outgoing traffic in the link(eth1), which leads
> to the internet in order to calculate the bandwidth usage. Eth1 link
> bandwidth capability is around 10 Mbps. we are able utilize a maximum
> of 7-8 Mbps when squid is running. After 15 minutes, there is a sudden
> drop in bandwidth from 8Mbps to 6.5 Mbps and it comes back to 8Mbps
> after 2 -3 min.
>
>
>>> Squid version : 2.6.STABLE14
>>
>> That is rather old (the last release of the 2.6 branch was STABLE23 September
>> 2009). Is there any reason you have not upgraded to a current version?
>>
>>
> =========
> There are some practical difficulties(our side) in upgrading to
> newer version.
>
>> Regards,
>>
>>
>> Antony.
>>
>> --
>> Behind the counter a boy with a shaven head stared vacantly into space,
>> a dozen spikes of microsoft protruding from the socket behind his ear.
>>
>> - William Gibson, Neuromancer (1984)
>>
>> http://www.Open.Source.IT Please reply to the list;
>> The Open Source IT forum please don't CC me.
