Hey Jérôme, I think it is good to understand the size of the cache array.. As I suggested before there is a limit to squid instances.. once you intercept connections you are in a pinch! the situation you are talking about is a bit different in case of reverse proxies and forward proxies. You also need to state the purpose of the proxies if it's intercept or tproxy. All the above matter in a way since in many cases it is better to use some routing machine to do the balancing rather then use CARP array clusters. A routing system can survive lots more traffic just by the kernel which has native SMP support by design.. All the above also takes in account that the linux OS has ip route cache function that makes the routing decisions smoother then smaller machines that cannot do the same.. So for an example: Router 1: 192.168.100.254/24+192.168.101.254/24+external/subnet machine 1:192.168.100.1/24+192.168.101.1/24 machine 2:192.168.100.2/24+192.168.101.2/24 etc.. a packet comes from the LAN and hits the Router 1(192.168.100.254) towards 122.122.122.200:80 The router either has LB route decision for the next hop and selects one of the 10 proxies IP(not machine but IP). the packet being routed to the next hop which is machine 1. machine 1 then in place intercepts the port 80 packet(iptables on the router can mark packets of port 80 dst only as marked for a proxy route) into port 3129. Then the proxy route for 0.0.0.0 is through 192.168.101.254 which on the router is marked as *outgoing* packet and routed using the external interface towards the destination machine while the connection is established between the client and the proxy. The session from the proxy towards the LAN session is being marked and routed back towards the proxy based on the route cache.. HERE this is where RP filtering is plays the good\bad guy :D now every packet that is marked as let say 3000 is towards the proxy and every 4000 mark is towards the external while 5000 is towards the LAN.. In the mean time the proxies uses HTCP to make sure that each and one of them has or doesn't have the relevant object(HTTP URL+HEADERS, unlike ICP that only applies the URL and can be in specific services the better and faster choice..). OK so this is almost the big picture on this small network.. so now the only limit is that squid suppose to serve about 150MBps which is about 1228800 bps if I am not wrong about it. OK so now there suppose to be a limit which is a bit odd.. SO how much RAM is needed for the routing system?? depends on the the network traffic and load... since every OS (linux) comes with a hard limit and softlimit that can be tunned it can be tested... Once we have a big system in hands with ram only we can use 10 machines as a balanced base line that we can test squid and linux to stretch them from 1GB which each machine takes the load of 100MB towards 200MB per machine.. a cluster of 5.. Just a reminder that haproxy can load 100,000 requests per sec on a core2quad or a core2duo machine on a testing environment. Squid is a much complex piece of software then haproxy taking in account tons of traffic each and every second towards the CPU and RAM. Prepare a DNS CACHE for this network...(squid also has a DNS resolution caching but only as good as needed for a HTTP proxy not.. BIND) So now the network is about: 1GBx4 on the core router. 1GBx2 cable between each and every proxy and server to the LAN switch(x10) 1GBx2 cable for the DNS cache server. Switch that can support this level of traffic which I think a simple 16x1GB ports of planet switch can lift.(the better is better)[is there a way to build a switch to all of that in less then 20k$?] once the lights are blinking you know that there is a dramatic network traffic.. The no DISK part is good for a very specific purpose of TESTING.. a cache that uses more then 2GB will probably need fast SAS 15k disks and no don't compare SSD to SAS since a spinning disk in a 15k rounds per sec is very fast.. You can instead do a test at home and use a small very very very FAST DOK to write a small fedora 4GB ISO.. compare that to a SAS drive.. Sibling proxy setup is better for the specific case but remember to add one more interface and network switch to the traffic between the proxy server traffic of only HTCP(We dont want HTTP headers to fly into the wrong hands of whoever comes to work in the network). I am sure someone will find this mailing list using a simple www.squid-cache.org and google and he can feel free to ask whatever he wants and there will be someone that will read it. Eliezer On 10/02/2013 12:02 PM, Jérôme Loyet wrote: > Hello, > > I'm facing a particular situation. I have to set-up a squid cluster on > 10 server. Each server has a lot of RAM (192GB). > > Is it possible et effective to setup squid to use only memory for > caching (about 170GB) ? What directive should be tweaked ? (cache_mem, > cache_replacement_policy, maximum_object_size_in_memory, ...). The > cache will store object from several KB (pictures) up to 10MB (binary > chunks of data). > > With 10 sibling cache squid instances, what should I use as type of > cache peering ? (10 siblings or a multicast ICP cluster, other ?) > > I've searched a little bit on google but didn't find anything revelant. > > Thanks to you all, > ++ Jerome >
