Dererk wrote:
Hi there!
I'm doing some experiments on caching HTTP requests in the middle of the
way between my webservers to the SOLR load balancer (lvs).
What I need to achieve is an extreme hot cache in the fastest possible
way. The trick is that objects don't live longer than 15 minutes :-(
This is because the "objects" (in Solr language, searches) are likely to
change between that 15 min period. Unfortunately, that's a requirement,
unless so far until I learn more how to write Squid plugins to play with
timeouts and stuff like that, which hope it's soon :-)
The interesting part is that the incoming traffic handled by my squid is
averaged 20Mbytes with peaks of 45Mbytes/s, with a hit ratio of 25 to
30%! It's awesome to see how great it behaves!
The reason why I send this message here, above from thank you about this
amazing software, is to see if I could get some tips I miss to suck the
most I could from Squid in the way to store/get objects as fast as
possible.
I've compiled Squid version 2.7.STABLE6 with this parameters:
'--prefix=/usr/local/squid' '--bindir=/usr/local/bin'
'--sbindir=/usr/local/sbin' '--sysconfdir=/etc/squid'
'--localstatedir=/var' '--mandir=/usr/local/man'
'--infodir=/usr/local/info'
'--disable-internal-dns'
Step 1: re-enable the parallel-threaded DNS client inside Squid.
'--enable-async-io'
'--enable-storeio=diskd,aufs,ufs,coss' '--with-aio' '--with-large-files'
'--enable-snmp' '--with-maxfd=8192'
And I attach my conf below.
Thanks in advance!
---
http_port serverPort:8983 accel vhost
http_port 3128
hierarchy_stoplist cgi-bin
acl QUERY urlpath_regex cgi-bin
no_cache deny QUERY
cache_mem 7168 MB
cache_swap_low 90
cache_swap_high 95
maximum_object_size 4096 KB
minimum_object_size 0 KB
cache_replacement_policy lru
memory_replacement_policy lru
You want one of the "heap "algorithms. Search for and Read about them
carefully then decide which is fastest for your traffic.
cache_dir diskd /var/cache/squid/vol1 10000 100 10
cache_dir diskd /var/cache/squid/vol2 10000 100 10
What OS are you running on? You enable the Linux-centric async-io / aio
but don't use AUFS to benefit from them...
NP: making the L1/L2 values "100 10" is probably a bad idea. AFAIK they
should be exponentials of 2 for fastest file path location.
For 10GB caches you probably want something like "32 256" or "256 32"
Step 3: (Assuming from some other config option hints that this is a
Squid-2.7)
add a COSS cache_dir for "small" objects (under about 256 KB). For
small stuff this is _much_ faster than the other disk types.
access_log /var/log/squid/access.log
cache_log /var/log/squid/cache.log
cache_store_log none
emulate_httpd_log on
log_ip_on_direct off
redirect_rewrites_host_header off
refresh_pattern . 0 20% 4320
With a 15 minute maximum age of objects you probably want the above to be:
refresh_pattern . 0 20% 900
cache_peer loadbalancer parent 8983 0 no-query originserver login=bleh
cache_mgr me@xxxxxx
cache_effective_user squid
cache_effective_group squid
coredump_dir /var/cache/squid
Greetings,
Dererk
Amos
--
Please be using
Current Stable Squid 2.7.STABLE6 or 3.0.STABLE18
Current Beta Squid 3.1.0.13
