nitesh naik wrote:
Hi All,
Issues was with Disk I/O. I have used null cache dir and squid
response is much faster now.
cache_dir null /empty
Thanks everyone for your help.
Regards
Nitesh
Oh dear, I can't believe I overlooked this.
cache_dir aufs (linux) or diskd (FreeBSD) is likely to solve the disk
speed issues. Particularly if you don't use a blocking IOEngine (leaving
it unset is usually best).
Amos
On Tue, Nov 4, 2008 at 9:40 AM, nitesh naik <niteshnaik@xxxxxxxxx> wrote:
Does these Redirector statistics mean url rewrite helper program is
slowing down squid response ? avg service time is 1550 msec.
Redirector Statistics:
program: /home/zdn/bin/redirect_parallel.pl
number running: 2 of 2
requests sent: 1069753
replies received: 1069752
queue length: 0
avg service time: 1550 msec
# FD PID # Requests Flags Time Offset Request
1 10 18237 12645 B 0.002 38 (none)
2 15 18238 12335 2.144 0 (none)
Regards
Nitesh
On Mon, Nov 3, 2008 at 2:46 PM, nitesh naik <niteshnaik@xxxxxxxxx> wrote:
Not sure if url rewrite helper is slowing down process because via
cache manager interface it didn't show any connection back log. What
information I should look for in cache manager to find out the cause
of the slow serving of requests ?
Redirector Statistics:
program: /home/zdn/bin/redirect_parallel.pl
number running: 2 of 2
requests sent: 155697
replies received: 155692
queue length: 0
avg service time: 0 msec
# FD PID # Requests Flags Time Offset Request
1 8 21149 104125
BW 0.033 38 http://s2.xyz.com/1821/78/570/1789/563/i88.js?z=4258
81.52.249.106/- - GET myip=10.0.0.165 myport=80\n
2 9 21150 51572 BW 0.039 0 http://s2.xyz.com/1813/2/570/1781/563/i7.js?z=8853
81.52.249.106/- - GET myip=10.0.0.165 myport=80\n
Following are my squid settings.
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1
acl to_localhost dst 127.0.0.0/255.0.0.0
acl localnet src 10.0.0.0/255.0.0.0
acl SSL_ports port 443
acl Safe_ports port 80 21 443 70 210 1025-65535 280 488 591 777
acl CONNECT method CONNECT
http_access Allow manager localhost
http_access Deny manager
http_access Deny !Safe_ports
http_access Deny CONNECT !SSL_ports
http_access Allow all
http_access Allow localnet
http_access Deny all
icp_access Allow localnet
icp_access Deny all
htcp_access Allow localnet
htcp_access Deny all
htcp_clr_access Deny all
ident_lookup_access Deny all
http_port 0.0.0.0:80 defaultsite=s1.xyz.com vhost
cache_peer 10.0.0.175 Parent 80 0 no-query round-robin originserver
cache_peer 10.0.0.177 Parent 80 0 no-query round-robin originserver
cache_peer 10.0.0.179 Parent 80 0 no-query round-robin originserver
cache_peer 10.0.0.181 Parent 80 0 no-query round-robin originserver
dead_peer_timeout 10 seconds
hierarchy_stoplist cgi-bin
hierarchy_stoplist ?
cache_mem 0 bytes
maximum_object_size_in_memory 1048576 bytes
memory_replacement_policy lru
cache_replacement_policy lru
cache_dir ufs /home/zdn/squid/var/cache 6000 16 256 IOEngine=Blocking
store_dir_select_algorithm least-load
max_open_disk_fds 0
minimum_object_size 0 bytes
maximum_object_size 4194304 bytes
cache_swap_low 90
cache_swap_high 95
logformat combined %>a %ui %un [%[tl] "%"rm %"ru HTTP/%">v" %Hs %<st
"%"{Referer}>h" "%"{User-Agent}>h" %Ss:%Sh
access_log /home/zdn/squid/var/logs/access.log squid
cache_log /home/zdn/squid/var/logs/cache.log
cache_store_log /home/zdn/squid/var/logs/store.log
logfile_rotate 10
emulate_httpd_log off
log_ip_on_direct on
mime_table /home/zdn/squid/etc/mime.conf
log_mime_hdrs off
pid_filename /home/zdn/squid/var/logs/squid.pid
debug_options ALL,1
log_fqdn off
client_netmask 255.255.255.255
strip_query_terms off
buffered_logs off
url_rewrite_program /home/zdn/bin/redirect_parallel.pl
url_rewrite_children 2
url_rewrite_concurrency 2000
url_rewrite_host_header off
url_rewrite_bypass off
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern (cgi-bin|\?) 0 0% 0
refresh_pattern . 0 20% 4320
quick_abort_min 16 KB
quick_abort_max 16 KB
quick_abort_pct 95
read_ahead_gap 16384 bytes
negative_ttl 0 seconds
positive_dns_ttl 21600 seconds
negative_dns_ttl 60 seconds
range_offset_limit 0 bytes
minimum_expiry_time 60 seconds
store_avg_object_size 13 KB
store_objects_per_bucket 20
request_header_max_size 20480 bytes
reply_header_max_size 20480 bytes
request_body_max_size 0 bytes
via off
ie_refresh off
vary_ignore_expire off
request_entities off
relaxed_header_parser on
forward_timeout 240 seconds
connect_timeout 10 seconds
peer_connect_timeout 5 seconds
read_timeout 120 seconds
request_timeout 10 seconds
persistent_request_timeout 120 seconds
client_lifetime 86400 seconds
half_closed_clients off
pconn_timeout 60 seconds
ident_timeout 10 seconds
shutdown_lifetime 30 seconds
cache_mgr webmaster
mail_program mail
cache_effective_user zdn
httpd_suppress_version_string off
umask 23
announce_period 31536000 seconds
announce_host tracker.ircache.net
announce_port 3131
client_persistent_connections off
server_persistent_connections off
persistent_connection_after_error off
detect_broken_pconn off
snmp_port 0
snmp_access Deny all
snmp_incoming_address 0.0.0.0
snmp_outgoing_address 255.255.255.255
icp_port 3130
htcp_port 0
log_icp_queries on
udp_incoming_address 0.0.0.0
udp_outgoing_address 255.255.255.255
icp_hit_stale off
minimum_direct_hops 4
minimum_direct_rtt 400
netdb_low 900
netdb_high 1000
netdb_ping_period 300 seconds
query_icmp off
test_reachability off
icp_query_timeout 4000
maximum_icp_query_timeout 2000
minimum_icp_query_timeout 5
background_ping_rate 10 seconds
mcast_icp_query_timeout 2000
icon_directory /home/zdn/squid/share/icons
global_internal_static on
short_icon_urls on
error_directory /home/zdn/squid/share/errors/templates
err_html_text
email_err_data on
nonhierarchical_direct on
prefer_direct off
incoming_icp_average 6
incoming_http_average 4
incoming_dns_average 4
min_icp_poll_cnt 8
min_dns_poll_cnt 8
min_http_poll_cnt 8
tcp_recv_bufsize 0 bytes
check_hostnames off
allow_underscore on
dns_retransmit_interval 5 seconds
dns_timeout 120 seconds
dns_defnames off
hosts_file /etc/hosts
ignore_unknown_nameservers on
ipcache_size 1024
ipcache_low 90
ipcache_high 95
fqdncache_size 1024
memory_pools on
memory_pools_limit 5242880 bytes
forwarded_for on
cachemgr_passwd XXXXXXXXXX shutdown config reconfigure offline_toggle
client_db on
refresh_all_ims off
reload_into_ims off
maximum_single_addr_tries 1
retry_on_error off
as_whois_server whois.ra.net
offline_mode off
uri_whitespace strip
coredump_dir /home/zdn/squid/var/cache
balance_on_multiple_ip on
pipeline_prefetch off
high_response_time_warning 0
high_page_fault_warning 0
high_memory_warning 0 bytes
sleep_after_fork 0
windows_ipaddrchangemonitor on
Regards
Nitesh
On Mon, Nov 3, 2008 at 10:27 AM, nitesh naik <niteshnaik@xxxxxxxxx> wrote:
Henrik / Amos,
Tried using these setting and I could see see delay in serving the
requests even for cached objects.
1225687535.330 5459 81.52.249.101 TCP_MEM_HIT/200 1475 GET
http://abc.xyz.com/3613/172/500/248/211/i5.js?z=9059 - NONE/-
application/x-javascript
1225687535.330 5614 81.52.249.100 TCP_MEM_HIT/200 8129 GET
http://abc.xyz.com/3357/172/211/4/1/i4.js?z=6079 - NONE/-
application/x-javascript
1225687539.661 12327 168.143.241.12 TCP_MISS/200 2064 GET
http://bc.xyz.com/2333/254/496/158/122/i17.js?z=3473 -
ROUNDROBIN_PARENT/10.0.0.181 application/x-javascript
Following are timeouts that I have set.
connect_timeout 10 seconds
peer_connect_timeout 5 seconds
read_timeout 2 minutes
request_timeout 10 seconds
icp_query_timeout 4000
and cache peer settings.
cache_peer 10.0.0.175 parent 80 0 no-query originserver round-robin
cache_peer 10.0.0.177 parent 80 0 no-query originserver round-robin
cache_peer 10.0.0.179 parent 80 0 no-query originserver round-robin
cache_peer 10.0.0.181 parent 80 0 no-query originserver round-robin
Regards
Nitesh
On Sun, Nov 2, 2008 at 1:11 AM, Henrik Nordstrom
<henrik@xxxxxxxxxxxxxxxxxxx> wrote:
On tor, 2008-10-30 at 19:50 +0530, nitesh naik wrote:
url rewrite helper script works fine for few requests ( 100 req/sec )
but slows down response as number of requests increase and it takes
10+ second to deliver the objects.
I'v run setups like this at more than thousand requests/s.
Is there way to optimise it further ?
url_rewrite_program /home/zdn/bin/redirect_parallel.pl
url_rewrite_children 2000
url_rewrite_concurrency 5
Those two should be the other way around.
url_rewrite_concurrency 2000
url_rewrite_children 2
Regards
Henrik
--
Please be using
Current Stable Squid 2.7.STABLE5 or 3.0.STABLE10
Current Beta Squid 3.1.0.1
