Search squid archive

Re: Configuring Squid to behave as fast as possible

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Dererk wrote:
Hi there!

I'm doing some experiments on caching HTTP requests in the middle of the
way between my webservers to the SOLR load balancer (lvs).

What I need to achieve is an extreme hot cache in the fastest possible
way. The trick is that objects don't live longer than 15 minutes :-(
This is because the "objects" (in Solr language, searches) are likely to
change between that 15 min period. Unfortunately, that's a requirement,
unless so far until I learn more how to write Squid plugins to play with
timeouts and stuff like that, which hope it's soon :-)

The interesting part is that the incoming traffic handled by my squid is
averaged 20Mbytes with peaks of 45Mbytes/s, with a hit ratio of 25 to
30%! It's awesome to see how great it behaves!

The reason why I send this message here, above from thank you about this
amazing software, is to see if I could get some tips I miss to suck the
most I could from Squid in the way to store/get objects as fast as
possible.

I've compiled Squid version 2.7.STABLE6 with this parameters: '--prefix=/usr/local/squid' '--bindir=/usr/local/bin'
'--sbindir=/usr/local/sbin' '--sysconfdir=/etc/squid'
'--localstatedir=/var' '--mandir=/usr/local/man'
'--infodir=/usr/local/info'


'--disable-internal-dns'

Step 1: re-enable the parallel-threaded DNS client inside Squid.


'--enable-async-io'
'--enable-storeio=diskd,aufs,ufs,coss' '--with-aio' '--with-large-files'
'--enable-snmp' '--with-maxfd=8192'

And I attach my conf below.

Thanks in advance!

---
http_port serverPort:8983 accel vhost
http_port 3128
hierarchy_stoplist cgi-bin
acl QUERY urlpath_regex cgi-bin
no_cache deny QUERY
cache_mem 7168 MB
cache_swap_low 90
cache_swap_high 95
maximum_object_size 4096 KB
minimum_object_size 0 KB
cache_replacement_policy lru
memory_replacement_policy lru

You want one of the "heap "algorithms. Search for and Read about them carefully then decide which is fastest for your traffic.

cache_dir diskd /var/cache/squid/vol1 10000 100 10
cache_dir diskd /var/cache/squid/vol2 10000 100 10

What OS are you running on? You enable the Linux-centric async-io / aio but don't use AUFS to benefit from them...

NP: making the L1/L2 values "100 10" is probably a bad idea. AFAIK they should be exponentials of 2 for fastest file path location.

For 10GB caches you probably want something like "32 256" or "256 32"


Step 3: (Assuming from some other config option hints that this is a Squid-2.7) add a COSS cache_dir for "small" objects (under about 256 KB). For small stuff this is _much_ faster than the other disk types.


access_log       /var/log/squid/access.log
cache_log /var/log/squid/cache.log
cache_store_log none
emulate_httpd_log on
log_ip_on_direct off
redirect_rewrites_host_header off
refresh_pattern .               0       20%     4320

With a 15 minute maximum age of objects you probably want the above to be:

  refresh_pattern .               0       20%     900

cache_peer loadbalancer parent 8983 0 no-query originserver login=bleh
cache_mgr me@xxxxxx
cache_effective_user squid
cache_effective_group squid
coredump_dir /var/cache/squid


Greetings,

Dererk


Amos
--
Please be using
  Current Stable Squid 2.7.STABLE6 or 3.0.STABLE18
  Current Beta Squid 3.1.0.13

[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux