Thank you very much.
Those stats look much better than the low peak ones. Though still not
Very close to the theoretical limits Adrian published for 2.7.
Some very marginal increases may be gained from re-ordering your
http_access lines that check for WindowsUpdate. Doing the src check
before the dstdomain check (left-to-right) will save a few cycles per
request.
so: http_access Allow windowsupdate ispros
becomes: http_access Allow ispros windowsupdate
cache_store_log can be set to 'none' for less time logging debug info
you generally don't need.
You may want to experiment with the collapsed_forwarding feature. It's
designed to reduce server-side network lags so should increase the
internal speeds but depends on higher hit ratios for best effect, which
at >40% you have.
That's all I can see right now that might provide any improvement at all.
Amos
Nyamul Hassan wrote:
Thank you Amos for your valuable input on this. Please find attached a
snapshot of peak hour traffic.
I'm also attaching the following graphs:
1. Cache Hit Rate
2. Client Request Rate
3. CPU IOWait
4. Service Timers
I'm also attaching a copy of my cache configuration. Looking at it, can
you suggest me if I can get any better performance than it is? I think
the IOWait is way too high, and I am using regular commodity SATA HDDs.
Any input would be greatly appreciated.
Regards
HASSAN
----- Original Message ----- From: "Amos Jeffries" <squid3@xxxxxxxxxxxxx>
To: "Nyamul Hassan" <mnhassan@xxxxxxx>
Cc: "Squid Users" <squid-users@xxxxxxxxxxxxxxx>
Sent: Monday, November 17, 2008 07:01
Subject: Re: Large ACLs and TCP_OUTGOING_ADDRESS
Hi,
I run squid in an ISP scenario. We have got two identically configured
squid caches being load balanced among 4,000 users over a 50 Mbps link.
The
system runs quite well, although not without the occassional hiccups.
But,
there is a complain from users about not being able to access some
websites
because of same external IP. For this, we configured the squid.conf to
have
ACLs for different user blocks of /24 and have them mapped through
different
external IPs on each of these boxes.
However, not all /24 blocks have the same number of users, and I also
have
lots of real IPs still lying unused. I thought about creating different
ACLs for every 5 or 8 users, and then map them to different external
IPs.
But, having them distributed in 8 IPs in each group would mean at least
500
separate ACLs and their corresponding TCP_OUTGOING_ADDRESS directives.
My question is, will this affect the performance of squid? Can squid
handle
this?
Depends on the ACL type. Squid should be able to handle many easily. of
the ACl you need; src is the fastest, next best is dstdomain, then
dst. So
for a marginal boost when combining on one line, put then in that order.
Just look for shortcuts as you go.
My servers are each running on Core 2 Duo 2.33 GHz, 8 GB of RAM, 5 HDDs
(1x80GB IDE for OS, 4x160GB SATA for cache), total 256GB Cache Store
(64GB
on each HDD). One of the server's stats are (taken at a very low user
count
time):
Thank you. We are trying to collect rough capacity info for Squid
whenever
the opportunity comes up. Are you able to provide such stats around peak
load for our wiki?
The info we collect can be seen at
http://wiki.squid-cache.org/KnowledgeBase/Benchmarks
Amos
Cache Manager menu
Squid Object Cache: Version 2.7.STABLE4
Connection information for squid:
Number of clients accessing cache: 2133
Number of HTTP requests received: 6213380
Number of ICP messages received: 1441542
Number of ICP messages sent: 1441550
Number of queued ICP replies: 0
Request failure ratio: 0.00
Average HTTP requests per minute since start: 11488.3
Average ICP messages per minute since start: 5330.7
Select loop called: 78705022 times, 0.412 ms avg
Cache information for squid:
Request Hit Ratios: 5min: 41.7%, 60min: 43.8%
Byte Hit Ratios: 5min: 17.5%, 60min: 16.9%
Request Memory Hit Ratios: 5min: 16.2%, 60min: 14.4%
Request Disk Hit Ratios: 5min: 44.2%, 60min: 43.6%
Storage Swap size: 241613712 KB
Storage Mem size: 4194392 KB
Mean Object Size: 35.25 KB
Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
HTTP Requests (All): 0.55240 0.55240
Cache Misses: 0.72387 0.68577
Cache Hits: 0.02899 0.02451
Near Hits: 0.64968 0.64968
Not-Modified Replies: 0.00000 0.00000
DNS Lookups: 0.00000 0.00000
ICP Queries: 0.00033 0.00035
Resource usage for squid:
UP Time: 32450.582 seconds
CPU Time: 5725.342 seconds
CPU Usage: 17.64%
CPU Usage, 5 minute avg: 23.55%
CPU Usage, 60 minute avg: 23.66%
Process Data Segment Size via sbrk(): 775752 KB
Maximum Resident Size: 0 KB
Page faults with physical i/o: 2
Memory usage for squid via mallinfo():
Total space in arena: 1937988 KB
Ordinary blocks: 1934155 KB 34179 blks
Small blocks: 0 KB 0 blks
Holding blocks: 35360 KB 8 blks
Free Small blocks: 0 KB
Free Ordinary blocks: 3832 KB
Total in use: 1969515 KB 100%
Total free: 3832 KB 0%
Total size: 1973348 KB
Memory accounted for:
Total accounted: 5661786 KB
memPoolAlloc calls: 882142632
memPoolFree calls: 850766245
File descriptor usage for squid:
Maximum number of file descriptors: 65536
Largest file desc currently in use: 8068
Number of file desc currently in use: 7035
Files queued for open: 4
Available number of file descriptors: 58497
Reserved number of file descriptors: 100
Store Disk files open: 289
IO loop method: epoll
Internal Data Structures:
6867535 StoreEntries
432110 StoreEntries with MemObjects
430724 Hot Object Cache Items
6854443 on-disk objects
Generated Mon, 17 Nov 2008 15:36:52 GMT, by cachemgr.cgi/2.7.STABLE4
Cache Manager menu
authenticate_cache_garbage_interval 3600 seconds
authenticate_ttl 3600 seconds
authenticate_ip_ttl 0 seconds
authenticate_ip_shortcircuit_ttl 0 seconds
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 116.193.170.25
acl localhost src 127.0.0.1
acl ispros_proxies src 116.193.170.24/255.255.255.254
acl proxy01 src 116.193.170.24
acl to_localhost dst 127.0.0.0/255.0.0.0
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 1025-65535
acl Safe_ports port 443
acl Safe_ports port 21
acl Safe_ports port 70
acl Safe_ports port 210
acl Safe_ports port 280
acl Safe_ports port 488
acl Safe_ports port 591
acl Safe_ports port 777
acl CONNECT method CONNECT
acl windowsupdate dstdomain download.windowsupdate.com
acl windowsupdate dstdomain www.download.windowsupdate.com
acl windowsupdate dstdomain wustat.windows.com
acl windowsupdate dstdomain c.microsoft.com
acl windowsupdate dstdomain .update.microsoft.com
acl windowsupdate dstdomain windowsupdate.microsoft.com
acl windowsupdate dstdomain crl.microsoft.com
acl windowsupdate dstdomain redir.metaservices.microsoft.com
acl windowsupdate dstdomain images.metaservices.microsoft.com
acl wuCONNECT dstdomain www.update.microsoft.com
acl ...........
...
...
...
acl ...........
acl apache rep_header Server ^Apache
http_access Allow manager localhost
http_access Allow manager proxy01
http_access Deny manager
http_access Deny !Safe_ports
http_access Deny CONNECT !SSL_ports
http_access Allow CONNECT wuCONNECT ispros
http_access Allow windowsupdate ispros
http_access Allow CONNECT wuCONNECT ggnn_real
http_access Allow windowsupdate ggnn_real
http_access Allow CONNECT wuCONNECT ggnn_pk64
http_access Allow windowsupdate ggnn_pk64
http_access Allow CONNECT wuCONNECT ggnn_pk128
http_access Allow windowsupdate ggnn_pk128
http_access Allow CONNECT wuCONNECT ggnn_pk256
http_access Allow windowsupdate ggnn_pk256
http_access Allow CONNECT wuCONNECT ggnn_pk512
http_access Allow windowsupdate ggnn_pk512
http_access Allow CONNECT wuCONNECT ggnn_pknight
http_access Allow windowsupdate ggnn_pknight
http_access Allow ...
...
...
...
http_access Allow ...
http_access Allow localhost
http_access Deny all
http_reply_access Allow all
icp_access Allow ispros_proxies
ident_lookup_access Deny all
reply_body_max_size 0 Allow all
follow_x_forwarded_for Deny all
acl_uses_indirect_client on
delay_pool_uses_indirect_client on
log_uses_indirect_client on
ssl_unclean_shutdown off
sslproxy_version 1
http_port 0.0.0.0:3128 transparent protocol=http
tcp_outgoing_address ...
...
...
...
tcp_outgoing_address ...
zph_mode off
zph_local 0
zph_sibling 0
zph_parent 0
zph_option 136
cache_peer ... Sibling 3128 3130 proxy-only
dead_peer_timeout 10 seconds
hierarchy_stoplist cgi-bin
hierarchy_stoplist ?
cache_mem 4294967296 bytes
maximum_object_size_in_memory 65536 bytes
memory_replacement_policy lru
cache_replacement_policy lru
cache_dir aufs /cachestore/cache1 65536 16 256
cache_dir aufs /cachestore/cache2 65536 16 256
cache_dir aufs /cachestore/cache3 65536 16 256
cache_dir aufs /cachestore/cache4 65536 16 256
store_dir_select_algorithm least-load
max_open_disk_fds 0
minimum_object_size 0 bytes
maximum_object_size 1073741824 bytes
cache_swap_low 90
cache_swap_high 95
update_headers on
access_log /var/log/squid/access.log squid
logfile_daemon /usr/lib/squid/logfile-daemon
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log
logfile_rotate 10
emulate_httpd_log off
log_ip_on_direct on
mime_table /etc/squid/mime.conf
log_mime_hdrs off
pid_filename /var/run/squid.pid
debug_options ALL,1
log_fqdn off
client_netmask 255.255.255.255
strip_query_terms on
buffered_logs off
netdb_filename /var/log/squid/netdb.state
ftp_user Squid@
ftp_list_width 32
ftp_passive on
ftp_sanitycheck on
ftp_telnet_protocol on
diskd_program /usr/lib/squid/diskd-daemon
unlinkd_program /usr/lib/squid/unlinkd
storeurl_rewrite_children 5
storeurl_rewrite_concurrency 0
url_rewrite_children 5
url_rewrite_concurrency 0
url_rewrite_host_header on
redirector_bypass off
location_rewrite_children 5
location_rewrite_concurrency 0
max_stale 604800 seconds
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
quick_abort_min 16 KB
quick_abort_max 16 KB
quick_abort_pct 95
read_ahead_gap 16384 bytes
negative_ttl 300 seconds
positive_dns_ttl 21600 seconds
negative_dns_ttl 60 seconds
range_offset_limit 0 bytes
minimum_expiry_time 60 seconds
store_avg_object_size 13 KB
store_objects_per_bucket 20
request_header_max_size 20480 bytes
reply_header_max_size 20480 bytes
request_body_max_size 0 bytes
via on
cache_vary on
broken_vary_encoding Allow apache
collapsed_forwarding off
refresh_stale_hit 0 seconds
ie_refresh off
vary_ignore_expire off
request_entities off
relaxed_header_parser on
server_http11 off
ignore_expect_100 off
forward_timeout 240 seconds
connect_timeout 60 seconds
peer_connect_timeout 30 seconds
read_timeout 900 seconds
request_timeout 300 seconds
persistent_request_timeout 120 seconds
client_lifetime 86400 seconds
half_closed_clients on
pconn_timeout 60 seconds
ident_timeout 10 seconds
shutdown_lifetime 30 seconds
cache_mgr ...
mail_from ...
mail_program mail
cache_effective_user squid
cache_effective_group squid
httpd_suppress_version_string off
visible_hostname ...
umask 23
announce_period 31536000 seconds
announce_host tracker.ircache.net
announce_port 3131
httpd_accel_no_pmtu_disc off
delay_pools 0
delay_initial_bucket_level 50
wccp_router 0.0.0.0
wccp_version 4
wccp2_rebuild_wait on
wccp2_forwarding_method 1
wccp2_return_method 1
wccp2_assignment_method 1
wccp2_service standard 0
wccp2_weight 10000
wccp_address 0.0.0.0
wccp2_address 0.0.0.0
client_persistent_connections on
server_persistent_connections off
persistent_connection_after_error off
detect_broken_pconn off
digest_generation on
digest_bits_per_entry 5
digest_rebuild_period 3600 seconds
digest_rewrite_period 3600 seconds
digest_swapout_chunk_size 4096 bytes
digest_rebuild_chunk_percentage 10
snmp_port 3401
snmp_access Allow snmp_local localhost
snmp_access Deny all
snmp_incoming_address 0.0.0.0
snmp_outgoing_address 255.255.255.255
icp_port 3130
log_icp_queries on
udp_incoming_address 0.0.0.0
udp_outgoing_address 255.255.255.255
icp_hit_stale off
minimum_direct_hops 4
minimum_direct_rtt 400
netdb_low 900
netdb_high 1000
netdb_ping_period 300 seconds
query_icmp off
test_reachability off
icp_query_timeout 0
maximum_icp_query_timeout 2000
minimum_icp_query_timeout 5
mcast_icp_query_timeout 2000
icon_directory /usr/share/icons
global_internal_static on
short_icon_urls off
error_directory /usr/share/errors/English
err_html_text nonhierarchical_direct on
prefer_direct off
ignore_ims_on_miss off
max_filedescriptors 65536
tcp_recv_bufsize 0 bytes
incoming_rate 30
check_hostnames on
allow_underscore on
dns_retransmit_interval 5 seconds
dns_timeout 120 seconds
dns_defnames off
hosts_file /etc/hosts
dns_testnames netscape.com
dns_testnames internic.net
dns_testnames nlanr.net
dns_testnames microsoft.com
ignore_unknown_nameservers on
ipcache_size 1024
ipcache_low 90
ipcache_high 95
fqdncache_size 1024
memory_pools on
memory_pools_limit 5242880 bytes
forwarded_for on
cachemgr_passwd disable shutdown offline_toggle
cachemgr_passwd XXXXXXXXXX all
client_db on
reload_into_ims off
maximum_single_addr_tries 1
retry_on_error off
as_whois_server whois.ra.net
offline_mode off
uri_whitespace strip
coredump_dir /var/cache
balance_on_multiple_ip on
pipeline_prefetch off
high_response_time_warning 0
high_page_fault_warning 0
high_memory_warning 0 bytes
sleep_after_fork 0
zero_buffers on
windows_ipaddrchangemonitor on
Generated Mon, 17 Nov 2008 15:48:58 GMT, by cachemgr.cgi/2.7.STABLE4
------------------------------------------------------------------------
------------------------------------------------------------------------
------------------------------------------------------------------------
------------------------------------------------------------------------
--
Please be using
Current Stable Squid 2.7.STABLE5 or 3.0.STABLE10
Current Beta Squid 3.1.0.2
