Сергей А. Андросов wrote:
Hi there,
There is some problems and I asking for advice for spread squid load and
increase perfomance.
Now we have near 4000 users in our university, and one squid proxy
server running on paravirtualized virtual machine (2 x 1.6Ghz Xeon E5310)
We use Squid Accounting system calling SAcc for traffic accounting. It's
generates squid configuration file and reloads squid when user is
banned. Caching is off and squid uses only for proxying and accounting.
There is NCSA authentication with passwd file for 4000 users, and 4000
ACL's generated in squid configuration file like this:
acl popovasi proxy_auth popovasi
http_access allow popovasi
http_access allow popovasi CONNECT
acl halenko proxy_auth halenko
http_access allow halenko
http_access allow halenko CONNECT
acl mamatovaa proxy_auth mamatovaa
http_access allow mamatovaa
http_access allow mamatovaa CONNECT
etc..
Wow. wow. This takes the prize for the year I think.
Do you have any hair or sanity left after working with that?
Internet connection is not so wide, only 4Mbps and, as I think, most
hard on server is working with huge ACL lists.
----------
What we need and what we can:
We need to spread squid load for better latency and responce to queries
and turn on caching because of tiny internet channel.
And so we have a couple of servers for new squid proxies.
We can use mysql_auth helper for authentication because of all user
database stored in mysql (but in clear-text, we working on it).
We need some expirience and advices for better ACL management. How we
can use one acl for all authorized users in passwd file? Can we get
perfomance increase using CARP and parent proxy? What the perfomance hit
will be if we will use gnu-regexp instead of built-in?
*Any* regexp, give huge performance downgrade. There are better
alternatives in most situations and some juggling to reduce the hit in
others where its needed.
Maybe we should change all user and traffic accounting management. Can
you advice on dynamic distributed accounting and authentication
solutions based on squid (I called it cache-grid :))?
Now, ask you for your expirience. Thanks for reply. Feel free to give
man and URL's for reading on this thread.
Sorry for my English.
Serg Androsov.
Okay. I think a few small additions may help out a lot here. Lets start
at the top.
If you want to help out the community could you please record your
current speed/hit statistics (squidclient mgr:info) before starting any
of this. And again after. We would be very interested in real-life
benefit examples.
Now the changes:
First,
Are you running a squid 2.6 stable 18? That release has the fastest
squid code out so far.
Second,
You will need a helper script to do the authentication better than
reloading squid. The helper script needs to accept "<id> <username>
<password>" on input and give out "<id> OK" or "<id> ERR" for accept/deny.
If you have a standard system such as LDAP , ActiveDirectory,
eDirectory, etc there are helpers already floating around for those.
You have two choices in setting it up here:
Option #1, full auth_* helper such as this:
auth_param basic program /etc/squid/helper/users.sh
auth_param basic children 10
auth_param basic concurrency 50
auth_param basic realm Your School Name or Realm
auth_param basic credentialsttl 2 seconds
authenticate_ip_ttl 10 minutes
authenticate_ttl 3 minutes
acl users proxy_auth REQUIRED
(with maybe different TTL to suit your setup)
References:
http://www.squid-cache.org/Versions/v2/2.6/cfgman/auth_param.html
http://www.squid-cache.org/Versions/v2/2.6/cfgman/authentical_ttl.html
http://www.squid-cache.org/Versions/v2/2.6/cfgman/authentical_ip_ttl.html
Or option #2, a custom external acl.
external_acl_type user_test
ttl=5 negative_ttl=0 concurrency=50 children=5
%LOGIN /etc/squid/helper/users.sh
(above all on one line)
acl users external user_test
References:
http://www.squid-cache.org/Versions/v2/2.6/cfgman/extrnal_acl_type.html
Then just this once:
http_access allow users
http_access deny all
When thats going, kill the system reloading squid. Your accounting
system may still be good for making the database which the helper uses.
But it no longer needed to touch squid.
That should cut down your performance issues with reloading, and many
ACL all in one cut.
Third,
You say caching turned off at present. You would do well (up to 30%
we think bandwidth savings and faster internet) just by turning it on.
Before any actual tuning.
References:
http://www.squid-cache.org/Versions/v2/2.6/cfgman/cache_dir.html
If you post your squid.conf we shall see what else is possible to be done.
Amos
--
Please use Squid 2.6STABLE17+ or 3.0STABLE1+
There are serious security advisories out on all earlier releases.
