Chris Robertson wrote:
Al - Image Hosting Services wrote:
Hi,
I hope that someone can help me. I have 3 servers running squid acting
as a web accelerator for a single http server. I have been having a
large amount of problems with the http server locking up and having
drive failures and squid seems to be a great solution. I was
pleasantly surprised at the performance of squid when I first
installed it. Even on the slowest of my servers, it seemed faster than
directly from the http server, but after I put the load back on the
slowest of my servers it was 0% idle.
Where is the CPU usage at, User, System or Wait?
Since this server had a second hard drive that was basically not being
used, I moved the squid cache over to it. I then did some googling and
I made some other changes and the server is now only 0% idle a few
times a day. I am really suprised at just how tunable squid is and it
was wonderful that a change in configuration could make such a
difference. But since the busy season is coming, I still think that
more can be done to tune it.
Here are the changes that I made to the squid.conf:
redirect_rewrites_host_header off
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow all
Not a good idea, limit the access to just the domains you are serving
where at all possible.
icp_access allow all
http_port 80 accel vhost
cache_peer 10.10.1.4 parent 80 0 no-query originserver
hierarchy_stoplist cgi-bin ?
cache_mem 16 MB
cache_mem is worth expanding, it lets more objects stay in memory and
skip the disk IO waiting times.
maximum_object_size_in_memory 128 KB
memory_replacement_policy heap GDSF
cache_replacement_policy heap GDSF
cache_dir ufs /var/squid/cache 200 16 256
maximum_object_size 2048 KB
cache_swap_low 90
cache_swap_high 98
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log
dump cache_store_log its a waste of disk IO unless you really need it
for something weird.
emulate_httpd_log on
buffered_logs on
acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
half_closed_clients off
icp_port 0
log_icp_queries off
coredump_dir /var/squid/cache
Are there any other changes that can be made to improve cpu usage?
I also did an install from source, since I am using NetBSD systems I
used pkgsrc to install it. There are several build options:
arp-acl
aufs
carp
coss
diskd
icmp
ipf-transparent
pam-helper
pf-transparent
snmp
ssl
unlinkd
I looked up what some of these are, but I am not sure how they are
used by squid or if I really need them. My thought is that if I
compiled squid without these options it could improve performance. I
wish there was more information on running squid on BSD systems
(especially recent info) and then maybe I could be sure what effect
compiling without an option would have. For example, I know what pam
is, but do I need it?
You are not using authentication, so could do without PAM.
I looked up what unlinkd does and I can see it running, but will it
help cpu usage to use it?
Using --enable-truncate "gives a little performance improvement, but may
cause problems when used with async I/O. Truncate uses more filesystem
inodes than unlink.." asynch I/O refers to diskd, aufs and coss. The
only way to see if that will lower your CPU usage is to try, but...
I don't see diskd running as a seperate process, so I don't know that
I am even using it.
Your cache_dir line specifies "ufs". To use diskd, you need to change
that line to...
cache_dir diskd /var/squid/cache 200 16 256
... with a couple more optional parameters to tune outstanding
requests. To use aufs use...
cache_dir aufs /var/squid/cache 200 16 256
Either one is likely to have a positive effect on your CPU utilization
as Squid will spend less time waiting for I/O. Turning off the
cache_store_log is probably not going to hurt anything. Making sure
your origin server gives proper expiry information and removing the
QUERY acl (and the "cache deny QUERY" line that relies on it) would
remove one more CPU sink.
That said BSD family of systems get more out of diskd than aufs in
current Squid.
The only one that I am sure that I need is ssl.
Any help would be greatly appreciated!
Best Regards,
Al
Chris
On top of all those adjustments if you still need a boots make sure you
are using the latest Squid-2.7. Those releases have the most performance
optimizations for high-throughput situations.
Amos
--
Please be using
Current Stable Squid 2.7.STABLE5 or 3.0.STABLE10
Current Beta Squid 3.1.0.2
