-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 28/09/2014 11:06 a.m., Ahmd wrote:
> Hi guys ,
>
>
>
> Hi all .
>
> i need best practise for squid version and recommended config for
> rock store and filesystem/mounting that give me maximum bandwith
> saving ?
>
> i tried the last version of 3.head , but it hanged after 10 minutes
> of working !!
>
>
>
> for now , im using 3.4.7 and its stable , and I have 20 G rams and
> want to save bandwitsh based on rams then I will use rock.
I see in your config file that you have allocated 20GB RAM to just the
cache_mem memory caching by *each* of the 2 workers.
That means the memory cache will use 40 GB RAM for objects, and approx
500MB for its index. On top of that Squid requires memory for other
caches and indexes, and active transactions.
Also in your config I see a comment about experiments to find out why
Squid slows down after a while. I have no doubt that the reason Squid
slows down is that it is starting to use the operating systems "swap
memory" / "virtual memory". This will not occur immediately, but only
after sufficient traffic has gone through to approximately half-fill
the cache_mem RAM with objects.
I suggest that you:
1) configure "cache_mem 2 GB" instead [A], then
**
2) wait until Squid memory cache shows at or near 100% filled,
2) wait until next peak traffic period,
3) look at how much RAM is spare during that highest point of traffic,
4) add approx 30% [B] of that spare RAM size to the cache_mem value,
5) *reconfigure* Squid (do not restart),
6) repeat from 2 until you are satisfied Squid is using RAM nicely
[A] 2 GB being a small value that should not get anywhere near
swapping but still allow fairly large amount of cached objects.
[B] 30% is calculated by taking roughly 80% (leaving 20% margin for
unexpected traffic bursts) of the spare RAM divided by workers count,
then subtracting an additional 10% for extra margin per-worker.
The goal here is just to get the cache_mem value a single step closer
to optimum without risking memory-swapping again. So do not expect or
try to reach optimum value on first attempt.
If you later add cache_dir or change the workers you will need to
repeat the above process from scratch and the total memory
requirements will have changed.
>
> But before all of that I want to make sure large rock is working
> fine and make sure that shafed memory & rock is not limited to 32
> kb before I go ahead.
>
>
>
> For now I will post my squid.conf file and want help the best
> optimization for squid.conf to save more bandwidth as it can:
>
>
>
> Here is my config:
>
> =======================
>
> [root@squidsmp ~]# squid -v
>
> Squid Cache: Version 3.4.7
>
> configure options: '--build=i486-linux-gnu' '--prefix=/usr'
> '--includedir=/include' '--mandir=/share/man'
> '--infodir=/share/info' '--sysconfdir=/etc'
> '--enable-cachemgr-hostname=drx' '--localstatedir=/var'
> '--libexecdir=/lib/squid' '--disable-maintainer-mode'
> '--disable-dependency-tracking' '--disable-silent-rules'
> '--srcdir=.' '--datadir=/usr/share/squid'
> '--sysconfdir=/etc/squid' '--mandir=/usr/share/man'
> '--enable-inline' '--enable-async-io=8'
> '--enable-storeio=ufs,aufs,diskd,rock'
> '--enable-removal-policies=lru,heap' '--enable-delay-pools'
> '--enable-cache-digests' '--enable-underscores'
> '--enable-icap-client' '--enable-follow-x-forwarded-for'
> '--enable-auth'
> '--enable-basic-auth-helpers=LDAP,MSNT,NCSA,PAM,SASL,SMB,YP,DB,POP3,getpwnam
>
>
,squid_radius_auth,multi-domain-NTLM' '--enable-ntlm-auth-helpers=smb_lm'
> '--enable-digest-auth-helpers=ldap,password'
> '--enable-negotiate-auth-helpers=squid_kerb_auth'
> '--enable-arp-acl' '--enable-esi' '--disable-translation'
> '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid'
> '--with-filedescriptors=131072' '--with-large-files'
> '--with-default-user=squid' '--enable-linux-netfilter'
> 'build_alias=i486-linux-gnu' 'CFLAGS=-g -O2 -g -Wall -O2'
> 'LDFLAGS=' 'CPPFLAGS=' 'CXXFLAGS=-g -O2 -g -Wall -O2'
> '--enable-ltdl-convenience'
>
>
>
>
>
> [root@squidsmp ~]# uname -a
>
> Linux squidsmp 3.7.5 #1 SMP Mon Sep 15 07:04:35 EDT 2014 x86_64
> x86_64 x86_64 GNU/Linux
>
>
>
>
>
> [root@squidsmp ~]# cat /etc/squid/squid.conf
>
> # Example rule allowing access from your local networks.
>
> # Adapt to list your (internal) IP networks from where browsing
>
> # should be allowed
>
> acl localnet src 10.0.0.0/8 # RFC1918 possible internal
> network
>
> acl localnet src 172.16.0.0/12 # RFC1918 possible internal
> network
>
> acl localnet src 192.168.0.0/16 # RFC1918 possible internal
> network
>
> acl localnet src fc00::/7 # RFC 4193 local private network
> range
>
> acl localnet src fe80::/10 # RFC 4291 link-local (directly
> plugged) machines
>
> acl mysubnet src xxxxxx
>
> acl xxx src xxx
>
> acl localip src xxx
>
> acl SSL_ports port 443
>
> acl Safe_ports port 80 # http
>
> acl Safe_ports port 21 # ftp
>
> acl Safe_ports port 443 # https
>
> acl Safe_ports port 70 # gopher
>
> acl Safe_ports port 210 # wais
>
> acl Safe_ports port 1025-65535 # unregistered ports
>
> acl Safe_ports port 280 # http-mgmt
>
> acl Safe_ports port 488 # gss-http
>
> acl Safe_ports port 591 # filemaker
>
> acl Safe_ports port 777 # multiling http
>
> acl CONNECT method CONNECT
>
> ######################################################
>
> ##############################################################
>
> http_access allow xxx
>
> http_access allow manager xx
"best practice" update for squid-3.2 and later is to move the manager
ACL check down below the "deny CONNECT ..." one.
Also, I see you have a series of manager ACL tests later in your
config. You may want to remove this one entirely, but see also the
comment down below in that set.
>
> http_access deny !Safe_ports
>
> http_access allow localnet
>
> # Deny CONNECT to other than secure SSL ports
>
> http_access deny CONNECT !SSL_ports
>
> #######################################################
>
> ###########SMP oPTIONS#####################
>
> dns_v4_first on
>
> # 3 workers, using worker #1 as the frontend is important
>
> workers 2
>
> #three workers
>
> ########################################################
>
> #cpu_affinity_map process_numbers=1,2,3,4 cores=2,4,6,8
>
> #http_port 127.0.0.1:400${process_number}
>
> #http_port ::1:66000
Two problems above:
* port numbers only go to 65535. 66000 is invalid.
* IPv6 addresses are written with []. So if you try this or something
similar again use syntax [::1]:3128
>
> #########################################
>
> visible_hostname netstream.ps
>
> ####################################################
>
> ####Filterring##############
>
> acl blockkeywords dstdomain "/etc/squid/koko.acl"
>
> http_access deny blockkeywords
>
> #################################################
>
> ##Regix filtering##########
>
> acl xacl dstdom_regex "/etc/squid/squid-regex.acl"
>
> http_access deny xacl
>
> ##############################
>
> cache_log /var/log/squid/cache.log
>
> #access_log stdio:/var/log/squid/access.log
>
> access_log /var/log/squid/access.log
>
> ###############################################
>
> http_port xxx:65000
>
> http_port xxxx:3129 tproxy
>
> ############################################################################
>
> http_access allow mysubnet
>
> ############################################################################
>
>
################
>
> cache_mgr a@a
>
> cachemgr_passwd a all
>
> #######################################################
>
> # add user authentication and similar options here
Best practice is to list ACL names left-to-right based on speed of the
test being run.
In recent Squid releases the manager ACL has become a regex pattern.
Which makes it sower than the localhost or localip "src" type tests.
You will get a small amount of performance gain by placing manager
test last on all these lines.
>
> http_access allow manager localhost
>
> http_access allow manager localip
>
> http_access allow manager mysubnet
>
> http_access allow mysubnet manager
The above two lines do the same thing. Also, up above there is an
"allow mysubnet" without any other restrictions. So neither of these
two " allow IF (mysubnet AND manager)" lines will be reached anyway.
>
> http_access deny manager
>
> #######################################################################
>
> cache_mem 20000 MB
>
> # ^^ per process mem size.
>
> maximum_object_size 20 MB
NP: only useful if there is disk cache (cache_dir) configured without
the max-size= parameter.
>
> maximum_object_size_in_memory 20 MB
>
> ##################################################
>
> # the frontend requires a different name to the backend
>
> #######################################################################
>
> cache_swap_low 90
>
> cache_swap_high 95
>
NP: cache_swap_* parameters apply only to disk caching. There is no
point setting these until you have a cache_dir configured. Also the
values above are the defaults.
> ###########################################################
>
> #########################################################
>
> quick_abort_pct 95
>
> fqdncache_size 65535
>
> ipcache_size 65535
>
> ###########################################################
>
> ipcache_low 98
>
> ipcache_high 99
>
> #####################################################
>
> ########### WCCP2 Config#############
>
> wccp2_router xxx.x.x.
>
> wccp2_rebuild_wait off
>
> wccp_version 2
>
> wccp2_forwarding_method 2
>
> wccp2_return_method 2
>
> wccp2_assignment_method 2
>
> wccp2_service dynamic 80
>
> wccp2_service_info 80 protocol=tcp flags=src_ip_hash priority=250
> ports=80
>
> wccp2_service dynamic 90
>
> wccp2_service_info 90 protocol=tcp flags=dst_ip_hash,ports_source
> priority=250 ports=80
>
> ######################################################
>
> cache_effective_user squid
>
> cache_effective_group squid
>
> #######################################################
>
> memory_replacement_policy heap GDSF
>
> cache_replacement_policy heap LFUDA
>
> #####################################################
>
> dns_nameservers xxxxx
>
> #######################################################
>
> http_access deny all
>
> ##############################
>
> refresh_pattern ^ftp: 1440 20% 10080
>
> refresh_pattern ^gopher: 1440 0% 1440
>
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
>
> refresh_pattern . 0 20% 4320
>
> ####################################################
>
> #####################################
>
> ####Slow after sometime tshooting############
>
> memory_pools off
>
> pconn_timeout 2 minutes
>
> persistent_request_timeout 1 minute
>
> read_ahead_gap 128 KB
>
> ###############################################
>
> #####rotating logs#####
>
> logfile_rotate 0
>
> ###########################################################################
>
> max_filedescriptors 131072
>
> ###########
>
Amos
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)
iQEcBAEBAgAGBQJUKNWYAAoJELJo5wb/XPRjDGUH/2dO4u+9TDO8x4jXYLpIOVas
Vfqkzj+JVD3NO9oxT4dpL4uFfipZRyiFxF9HkOWE2tluYLLvEpOrI7BeCpt8dQzt
jKp/kX2Os6YLVxnkHOnffVJz3UtfS/SQaIXYzyT7rfne6bIsQHh11aCQcNTN/Vw6
z0YX6ii/TJifqxkL1RX9SBU3isEk2aO8bB98nu6VNX8hSjpr6fi3Tkw3kBjIyk8C
ggXQpS7Lsd0MIADxEvXu/nVQaM6qUdpSbDfUyWPmZzNmvo6rurn9IOJ7pLtx0rLK
OHQ8dNACo9kI8xSMEhTtEeDqykOKnrvka5yn5X/HF7gjFJm5wSE4jjOZNfvpbFM=
=4Zxt
-----END PGP SIGNATURE-----
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
