Markus Meyer wrote:
Hi folks,
I'm testing a CARP setup and try to find out what the maximum throughput
is. After a lot of testing I'm stuck with 5000 req/s and ca. 220 mbit/s
in and out. Whatever I do I can't make it faster. Does anyone have an
idea whats going on or if I do something wrong? How can I find out what
the bottleneck is?
5,000rps is around the fastest I have heard for Squid in production.
220mbit/s is about twice the bandwidth that user had. So well done on that.
This is the test setup:
- 1 GBit NICs and even faster switches, all servers in the same subnet
- 1 CARP and 5-9 Proxies as parents
- all proxies use AUFS and cache_mem
- proxies are warmed up, meaning I ran the test multiple tests so that
all content is on the proxies and they don't have to get anything from
the original servers
- the CARP does not cache anything(cache_mem 0 MB & cache_dir null no-store)
- used Version: Squid 2.7.STABLE7-2
- Kernel is the standard Etch 2.6.18
The test itself:
I use http_load[1] and start it on four servers at the same time with 50
parallel requests. It uses a list with appr. 3 mio. URLs with most of
the files being smaller than 4 kB but bigger than 2 kB. The test runs 30
minutes.
The CARP uses appr. 25% CPU and has lots of free memory. I get no
problems reported in cache.log or syslog. And the throughput of 5k req/s
and 220 mbit/s doesn't change with five, seven or nine parent proxies.
[1] http://www.acme.com/software/http_load/http_load-12mar2006.tar.gz
Any ideas and hints are welcome.
Cheers, Markus
Besides kinkies multi-instance suggestion the things I know of that you
have not mentioned are:
* turn zero_buffers off
* make sure epoll or kqueue is in use.
* experiment with the incoming_rate.
* tune the kernel TCP timeouts
* debugging in Squid is known to slow things down "debug_options ALL,0"
is the fastest.
* some small cache_mem on the CARP may or may not improve the overall
speed. It will on the hottest objects.
I assume there are no filtering or other access controls on the CARP for
limit testing?
If thats not fast enough for you, you could try the latest 2.HEAD and
3.HEAD for the latest speed boosters we have. 2.HEAD has the last of the
Yahoo! and Adrian speed experiments and is usably stable. 3.HEAD is
where we are working now, but may not be as fast as 2.x.
Amos
--
Please be using
Current Stable Squid 2.7.STABLE7 or 3.0.STABLE23
Current Beta Squid 3.1.0.16
