Search squid archive

Re: squid accel peer load balancing weighted round robin?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Sylvain Viart wrote:
Hi Chris,
Chris Robertson a écrit :
I've a squid Version 2.6.STABLE16
with the following config accel mode design:

proxy ----> php-01..php-08

php are apache server no slibing proxy.

If things haven't changed, weight only works with ICP. Basically an ICP query is sent out, and the weight is added to an internal calculation based on response latency (and possibly other factors) to determine which affirmative responding ICP peer to request the actual object from.
Ok, that what I suppose too. weight seems to be for ICP slibing proxy.
What I want effectively is, weighted round robin on origin server.

And it seems, that perform, round bobin as expected. But it may also be disturbed by some other peer in my definition. Here is the complement:

I define 8 peer, for php content, and 1 for static content, as you can see on the max-conn peer doesn't allow the same load. I first try to modify the weight introducing the strange behavior I described. The config I show is my new test, with url_rewriter weighted round robin. Which seems to work as expected.

# 8 php peers
cache_peer php-01 parent 80 0 no-query originserver login=PASS max-conn=50
cache_peer php-03 parent 80 0 no-query originserver login=PASS max-conn=100
cache_peer php-04 parent 80 0 no-query originserver login=PASS max-conn=50
cache_peer php-05 parent 80 0 no-query originserver login=PASS max-conn=70
cache_peer php-06 parent 80 0 no-query originserver login=PASS max-conn=70
cache_peer php-07 parent 80 0 no-query originserver login=PASS max-conn=120
cache_peer php-08 parent 80 0 no-query originserver login=PASS max-conn=100
cache_peer php-09 parent 80 0 no-query originserver login=PASS max-conn=180

# 1 peer for admin content,
cache_peer php-02 parent 80 0 no-query originserver login=some:thing max-conn=10

# 1 peer for static content
cache_peer filer-vip parent 80 0 no-query originserver

# acces content filtering based on acl bellow
cache_peer_access filer-vip allow static_doc
cache_peer_access filer-vip deny all

# acces content filtering based on acl bellow
cache_peer_access php-01 allow php_01
cache_peer_access php-01 deny all
cache_peer_access php-03 allow php_03
cache_peer_access php-03 deny all
cache_peer_access php-04 allow php_04
cache_peer_access php-04 deny all
cache_peer_access php-05 allow php_05
cache_peer_access php-05 deny all
cache_peer_access php-06 allow php_06
cache_peer_access php-06 deny all
cache_peer_access php-07 allow php_07
cache_peer_access php-07 deny all
cache_peer_access php-08 allow php_08
cache_peer_access php-08 deny all
cache_peer_access php-09 allow php_09
cache_peer_access php-09 deny all

# third acces content filtering based on other acl, not shown
cache_peer_access php-02 allow authorized_client admin_doc
cache_peer_access php-02 deny all

# here is the acl needed to work
acl php_01 urlgroup php01
acl php_03 urlgroup php03
acl php_04 urlgroup php04
acl php_05 urlgroup php05
acl php_06 urlgroup php06
acl php_07 urlgroup php07
acl php_08 urlgroup php08
acl php_09 urlgroup php09

acl static_doc dstdomain filer-vip

and basically what the  the redirector do is:
based on regexp split the URL in 2 groups, filer URL and PHP url.

filer URL are rewrited with the pseudo domain 'filer-vip' which is matched against static_doc acl. php URL are rewrited and canonized, based on dstdomain and url path regexp. Next the url are prefixed by a urlgroup tag. Here based on peer round robbin.

Which gives the perl loop:

not extensively tested right now but works better than squid weight.

while (<>)
{
       # match filer URL
if(m[http://($filer_host)/] || m[http://([^/]+)/(js/static_file|media|thumb)])
       {
               s#http://[^/]+(:[0-9]+)?#http://$filer#;

               # add urlgroup
               s#http://#!filer!$&#;
       }
       else
       {
               # if is a php multi domain, and not a dynamic content
               # rewrite it in cannonical form

               # the first regexp extract the domaine name in $1
               if(m[http://([^/]+)])
               {
                       # extract domain name from the input
                       $domain = $1;
               }

# if the domain is not norewrite_domain and the content is not a script if($domain !~ /^($norewrite_domain)$/ && $_ !~ /\.($script_ext)/)
               {
                       s#http://[^/]+(:[0-9]+)?#http://$php#;
               }

               # pickup a round robbin server
               $originserver = $php[($n_php++%$nb_php)];
               # add urlgroup
               s#http://#!php$originserver!$&#;
       }

       # print the rewrited line
       print;
}


Squid 3's conf file seems to confirm this (http://www.squid-cache.org/Versions/v3/3.0/cfgman/cache_peer.html):

use 'weight=n' to affect the selection of a peer during any weighted peer-selection mechanisms. The weight must be an integer; default is 1, larger weights are favored more. This option does not affect parent selection if a peering protocol is not in use.
Seems also for ICP proxy, even weighted-round-robin, an squid3 option, seems to be related to ICP proxy.

If you are going by the squid.conf docs on that, they are still wrong in some details of these selection algorithms. Its getting updated as we find bits.

cache_peer php-03 parent 80 0 no-query originserver round-robin weight=1
cache_peer php-04 parent 80 0 no-query originserver round-robin weight=1
cache_peer php-05 parent 80 0 no-query originserver round-robin weight=1
cache_peer php-06 parent 80 0 no-query originserver round-robin weight=1
cache_peer php-07 parent 80 0 no-query originserver round-robin weight=3
cache_peer php-08 parent 80 0 no-query originserver round-robin weight=1

Strangely the weight seems not impact the load balancing as I might expect.

If increase weight on a the peer php-03 result
 * less load on peer 01
 * more load on peer 03
 * less load on peer 04
 * more load on peer 05
 * same load on peer 06
 * same load on peer 07
 * more load on peer 08

All my test seem to show always very more load on the last peer in the list. That why I protect with max-conn=100.

How are you measuring load? Round-robin is exactly what it sounds like. Each request is sent to the next peer in line. Processing load is not taken in to consideration.

I measure load by looking on some MRTG like graph of all the server pool. And clearly it see, than the load is badly divided on each peer. But if it's supposed to work with slibing proxy, I can live with that somewhat bug. :-)

config problem.

use "round-robin" for strictly old fashioned round-robin, "weighted-round-robin" for round-robin with "weight=" load balancing

>
CARP is purpose build load balancing algorithm, and as far as I know, it should work with originserver. http://docs.huihoo.com/gnu_linux/squid/html/x2398.html
>
Yep, I saw it too. Seems to be also for slibing proxy, no?

No. It's a parent proxy/server thing.
"
use 'carp' to define a set of parents which should
be used as a CARP array. The requests will be
distributed among the parents based on the CARP load
balancing hash function based on their weight
"
says so twice to be sure.

FWIW, "originserver" only affects the replies squid produces. Whether it spoofs being a web server for the data requested.

Amos

[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux