J. Webster wrote:
What is the best place to start with in cache analysis?
Would it be cache size, memory object size, IO, etc.?
I'm looking to optimise the settings for my squid server.
Step 0) migrate to the latest Squid 2.7 or 3.1 or if possible 2.HEAD
(that one is only nominally beta, it's very stable in reality)
1) Start by defining 'optimize' ... are you going to prioritize...
Faster service?
More bandwidth saving?
More client connections?
2a) For faster service, look at DNS delays, disk IO delays, maximizing
cacheable objects (dynamic objects etc).
2b) For pure bandwidth savings start with a look at object cacheablity.
Check dynamics are being cached, ranges are being fetched in full, etc
3) Then profile all the objects stored over a reasonably long period,
looking at size. compare with the age of objects being discarded.
3a) tune the storage limits to prioritize the storage locations. giving
priority to RAM, then COSS, then AUFS/diskd.
3b) set the storage limits as high as possible to maximize amount of
data stored. anywhere.
4) take a good long look at your access controls and in particular the
types speedy/fast/slow. You may get some speed benefits from fixing up
the ordering a bit. regex are killers, remote lookups (helpers, or DNS)
are second worst.
(some performance hints below)
5) repeat from (2b) as often as possible. concentrate traffic which
seems to logically be storeable but gets a TCP_MISS anyway.
Objects served from cache lead to faster service ties for those objects,
so the speed vs bandwidth are inter-related somewhat. But there is a
tipping point somewhere where tuning one starts to impact the other.
Server: about 220GB available for the cache, I'm only using 40000 MB at present as in the config below.
system D2812-A2
/0 bus D2812-A2
/0/0 memory 110KiB BIOS
/0/4 processor Intel(R) Core(TM)2 Duo CPU E7300 @ 2.66GHz
/0/4/5 memory 64KiB L1 cache
/0/4/6 memory 3MiB L2 cache
/0/4/0.1 processor Logical CPU
/0/4/0.2 processor Logical CPU
/0/7 memory 3MiB L3 cache
/0/2a memory 1GiB System Memory
/0/2a/0 memory 1GiB DIMM DDR2 Synchronous 667 MHz (1.5 ns)
/0/2a/1 memory DIMM DDR2 Synchronous 667 MHz (1.5 ns) [empty]
/0/2a/2 memory DIMM DDR2 Synchronous 667 MHz (1.5 ns) [empty]
/0/2a/3 memory DIMM DDR2 Synchronous 667 MHz (1.5 ns) [empty]
/0/1 processor
/0/1/0.1 processor Logical CPU
/0/1/0.2 processor Logical CPU
Current squid.conf:
---------------------
auth_param basic realm Proxy server
auth_param basic credentialsttl 2 hours
auth_param basic program /usr/lib/squid/ncsa_auth /etc/squid/squid_passwd
authenticate_cache_garbage_interval 1 hour
authenticate_ip_ttl 2 hours
acl all src 0.0.0.0/0.0.0.0
all src all
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl localhost src 127.0.0.1
acl cacheadmin src 88.xxx.xxx.xxx
acl to_localhost dst 127.0.0.0/8
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 1863 # MSN messenger
acl ncsa_users proxy_auth REQUIRED
acl maxuser max_user_ip -s 2
acl CONNECT method CONNECT
http_access allow manager localhost
http_access allow manager cacheadmin
Hint: add the localhost IP to the cacheadmin ACL and drop one full set
of "allow manager localhost" tests.
http_access deny manager
http_access allow ncsa_users
Hint: drop the authentication down ...
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny to_localhost
... to here. All the attacks against your proxy for bad ports and
sources will be dropped quickly by the security blanket settings. Load
on your auth server will reduce and may speed up it's response time.
Hint 2: if possible, define an ACL or the network ranges where you
accept logins. Use it like so:
http_access allow localnet ncsa_users
... once again that speeds up the rejections, and helps by reducing
the number of times the slow auth lookup needs checking.
http_access deny maxuser
http_access allow localhost
If localhost really is allowed to do anything, move it up above the
"to_localhost" one.
Otherwise drop this completely, having the correct auth login details
will permit links from localhost just as easily as from anywhere else.
http_access deny all
icp_access allow all
Define the networks where peer siblings are trusted. Allwo them and deny
everything else.
That will reduce a fair bit of load on your Squid trying to service
random ICP requests from the general Internet.
http_port 8080
http_port 88.xxx.xxx.xxx:80
hierarchy_stoplist cgi-bin ?
cache_mem 100 MB
Bump this up as high as you can go without risking memory swapping.
Objects served from RAM are 100x faster than objects not.
maximum_object_size_in_memory 50 KB
cache_replacement_policy heap LFUDA
cache_dir aufs /var/spool/squid 40000 16 256
If you pick 2.x squid to upgrade to, add a COSS directory as well.
See the recent threads on optimizing COSS for how to tune that.
maximum_object_size 50 MB
Bump this up too. Holding full ISO CDs and windows service packs can
boost performance when one is used from the cache. 40GB of disk can
store a few.
cache_swap_low 90
cache_swap_high 95
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
buffered_logs on
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
Drop the QUERY bits above. It's more than halving the things your Squid
can store.
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
Add right here:
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
half_closed_clients off
cache_mgr aaa@xxxxxxx
cachemgr_passwd aaa all
visible_hostname ProxyServer
log_icp_queries off
dns_nameservers 208.67.222.222 208.67.220.220
hosts_file /etc/hosts
memory_pools off
Might cause efficiency problems if the underlying malloc is not
optimized. but oh well, up to you.
forwarded_for off
client_db off
coredump_dir /var/spool/squid
Amos
--
Please be using
Current Stable Squid 2.7.STABLE7 or 3.0.STABLE23
Current Beta Squid 3.1.0.16
