Search squid archive

Re: Architecture

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Ronan Lucio wrote:
Hi Amos,

Thank your for the valuable answer.
I'm sorry for the delayed reply, but I need to first read more about CARP and architecture to better digest it.

Now these things seems to be getting clear in my mind.

Amos Jeffries escreveu:
What kinkie means is that the efficiency is determined by the type of NAS.

Squid performs a high-churn random-access IO when in full operation. And
needs to do so at the highest possible speed. The absolute _best_ solution
is to have a high speed storage device spindle dedicated to just one
cache_dir in one Squid. None of the SAN/NAS I'm aware of can make that kind
of find-grained assignment.

Actually, only the application servers and the third (lighttpd for static... if so) would have access to the storage.
Squid servers would read and write cached file localy in a SAS HD.

Almost...

1a) Squid load balancer using carp to select layer-2.

1b) Squid caching servers

then whatever backend you like...


It really seems to be a better choice.
Do you have any idea about how many page hit would handle one squid servers? Thinking about a Dual QuadCore 4Gb RAM serving only small files (less than 300 Kb each).

Adrian has mapped Squid 2.7 as far as 800-850 req/sec. Squid-3 is untested but expected to be at minimum 500 req/sec (2.5 was capable of this around the time of the Squid-3 branch).
NP: that is on single-threaded machines.

We have confirmed data from one Squid-2.7 install doing 2.7K req/sec on high-end quad-core hardware a few years back and unconfirmed reports of another deployment (unknown version) topping 5K req/sec 1-2 years ago.

Wikimedia are the poster deployment for CARP deployment to around ~200 Squid last I heard and push dozens of TB per day (their network diagrams show 3 load balancing Squid, I'm not sure if thats right though). So even if one Squid can't handle it the model scales enormously. Their case-study has graphs from back when it was 50 Squid show 560+ req/sec ...
http://www.squid-cache.org/Library/wikimedia.dyn


Chris Robertson had some numbers a while back, he may chime in with something more accurate :)


Given the file sizes you mention, I'd suggest using caching layer Squid 2.7 with COSS involved as well.

Amos


2) Application server as the backend servers

3) A third server serving static resources

It's up to you whether the system is large enough to require separate (2)
and (3).
For small enough (for some value of small) its sufficient to combine them
and provide caching headers to push most of the static work into the (1b)
Squid.

Hmmm, the doubt remains.
I'm thinking in that third server for two reasons:
- Get out these apache process (~ 30 hits) from the application servers, once the goal is to optimize its resources. - Get a better performance assigning it (images, css's and js's) a different sub-domain, that would allow client browser to make 4-6 parallel requests.

Thank you,
Ronan

It's up to you. I just pint out that the whole caching model is to convert generated content into static and server that from Squid caches as close to the client as possible. The load on the static server will or should be extremely low by comparison to the dynamic servers.

The load reduction on static info servers with long'ish caching times starts high and approaches 100%. Which makes a dedicated static content server possibly a waste of a good CPU :)

These are dynamic image servers...
http://www.squid-cache.org/Library/flickr.dyn
The 'Layer-7 URL hashing' they mention is CARP.


Amos
--
Please be using
  Current Stable Squid 2.7.STABLE6 or 3.0.STABLE16
  Current Beta Squid 3.1.0.8

[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux