Hi again,
I am using one box (4Gb RAM, modern multicore CPU) for a mono-instance
proxy-only (non-caching) squid 3.1.9 version, serving about 2500
clients .
CPU is never over 30%, and vmstat output does not show any swapping.
1) The configuration of the instance is very simple indeed:
# ----------------------------------
# Recommended minimum configuration:
# Recommended access permissions, acls, ... etc...
........
........
#
# We recommend you to use at least the following line.
hierarchy_stoplist cgi-bin ?
cache_dir aufs /san/cache 241904 64 512 <- NOt used
cache deny all
NP: with a cache_dir configured it *is* used. The "cache deny all" means
that new data will not be added AND that existing data will be removed
when detected as stale or a variant collision.
# Add any of your own refresh_pattern entries above these.
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
# More rules
cache_mem 512 MB
# ----------------------------------
2) TCP is also tuned for performace, timeout optimization, etc..
3) On a high load moment I invoke the following command.
squidclient -p 8080 mgr:60min
which shows:
sample_start_time = 1294827593.936461 (Wed, 12 Jan 2011 10:19:53 GMT)
sample_end_time = 1294831195.510801 (Wed, 12 Jan 2011 11:19:55 GMT)
client_http.requests = 176.550847/sec
client_http.hits = 0.000000/sec
client_http.errors = 0.337075/sec
client_http.kbytes_in = 401.282290/sec
client_http.kbytes_out = 4950.528662/sec
client_http.all_median_svc_time = 0.121063 seconds
client_http.miss_median_svc_time = 0.121063 seconds
client_http.nm_median_svc_time = 0.000000 seconds
client_http.nh_median_svc_time = 0.000000 seconds
client_http.hit_median_svc_time = 0.000000 seconds
server.all.requests = 175.215042/sec
server.all.errors = 0.000000/sec
server.all.kbytes_in = 4939.614269/sec
server.all.kbytes_out = 226.921319/sec
server.http.requests = 167.784681/sec
server.http.errors = 0.000000/sec
server.http.kbytes_in = 4441.452123/sec
server.http.kbytes_out = 167.159676/sec
server.ftp.requests = 0.033319/sec
server.ftp.errors = 0.000000/sec
server.ftp.kbytes_in = 24.191365/sec
server.ftp.kbytes_out = 0.001944/sec
server.other.requests = 7.397043/sec
server.other.errors = 0.000000/sec
server.other.kbytes_in = 473.970780/sec
server.other.kbytes_out = 59.759977/sec
icp.pkts_sent = 0.000000/sec
icp.pkts_recv = 0.000000/sec
icp.queries_sent = 0.000000/sec
icp.replies_sent = 0.000000/sec
icp.queries_recv = 0.000000/sec
icp.replies_recv = 0.000000/sec
icp.replies_queued = 0.000000/sec
icp.query_timeouts = 0.000000/sec
icp.kbytes_sent = 0.000000/sec
icp.kbytes_recv = 0.000000/sec
icp.q_kbytes_sent = 0.000000/sec
icp.r_kbytes_sent = 0.000000/sec
icp.q_kbytes_recv = 0.000000/sec
icp.r_kbytes_recv = 0.000000/sec
icp.query_median_svc_time = 0.000000 seconds
icp.reply_median_svc_time = 0.000000 seconds
dns.median_svc_time = 0.030792 seconds
unlink.requests = 0.000000/sec
page_faults = 0.000278/sec
select_loops = 7354.368257/sec
select_fds = 6158.774443/sec
average_select_fd_period = 0.000000/fd
median_select_fds = 0.000000
swap.outs = 0.000000/sec
swap.ins = 0.000000/sec
swap.files_cleaned = 0.000000/sec
aborted_requests = 3.468483/sec
syscalls.disk.opens = 0.187418/sec
syscalls.disk.closes = 0.185474/sec
syscalls.disk.reads = 0.000000/sec
syscalls.disk.writes = 0.101622/sec
syscalls.disk.seeks = 0.000000/sec
syscalls.disk.unlinks = 0.101622/sec
syscalls.sock.accepts = 73.385407/sec
syscalls.sock.sockets = 63.842636/sec
syscalls.sock.connects = 63.166821/sec
syscalls.sock.binds = 63.842636/sec
syscalls.sock.closes = 102.078415/sec
syscalls.sock.reads = 3467.104333/sec
syscalls.sock.writes = 2584.626089/sec
syscalls.sock.recvfroms = 20.185062/sec
syscalls.sock.sendtos = 12.178285/sec
cpu_time = 709.070205 seconds
wall_time = 3601.574340 seconds
cpu_usage = 19.687785%
So, the system is serving 175 Request per Second (60 min. average), if I
am understanding well the output.
Well ... client_http.requests = 176.550847/sec
4) Looking the actual connections, I found (extracted from netstat
output)
8 CLOSE_WAIT
3 CLOSING
4216 ESTABLISHED
100 FIN_WAIT1
24 FIN_WAIT2
45 LAST_ACK
5 LISTEN
43 SYN_RECV
46 SYN_SENT
2273 TIME_WAIT
there are many connections, but not so many that cannot be handled for
the system, I guess.
However, network throughput is about 50 Mbps. Let us look, iptraf output
(general statistic for my 1 Gb eth0 interface):
Total rates: 92146,8 kbits/sec and 13393.6 packets/sec
Incoming rates: 43702,1 kbits/sec and 7480 packets/sec
Outgoing rates: 48464,k ktis/sec and 5913.6 packets/sec
The users, however, does not get a really good navigation experience
when reaching these values (on high load moments).. and I am not able to
discover where the bottleneck is (Squid itself, SO, network, drivers,
parameter tuning)..
Any idea on how to get a better throughput with this equipment. Any idea
about subsytems configuration, or general configuration of squid
software?
3.1.10 may be worth trialling. We have just had some surprising
benchmarks submitted for it. On a basic config it seems to perform a few
dozen % RPS faster than 3.1.9. Reason unclear, but the big change
between them was better memory management and better validation support.
I say trial because under high load the benchmarker found some CPU
problems we have not yet isolated and fixed. I hope you will experience
a similar improvement without the problems.
Indeed a great improvement when going from 3.1.9 to 3.1.11 and
activating caching. Now we get 170.000 kbit/sec under a heavy load.
Behaviour is much more predictable and stable. However, it is clear that
the CPU comsuption is higher. Now, it is near 30%, before, it was weird
we reached 10%.
Thank you for your great work,
--
Víctor Hernández
