Does these Redirector statistics mean url rewrite helper program is slowing down squid response ? avg service time is 1550 msec. Redirector Statistics: program: /home/zdn/bin/redirect_parallel.pl number running: 2 of 2 requests sent: 1069753 replies received: 1069752 queue length: 0 avg service time: 1550 msec # FD PID # Requests Flags Time Offset Request 1 10 18237 12645 B 0.002 38 (none) 2 15 18238 12335 2.144 0 (none) Regards Nitesh On Mon, Nov 3, 2008 at 2:46 PM, nitesh naik <niteshnaik@xxxxxxxxx> wrote: > Not sure if url rewrite helper is slowing down process because via > cache manager interface it didn't show any connection back log. What > information I should look for in cache manager to find out the cause > of the slow serving of requests ? > > Redirector Statistics: > program: /home/zdn/bin/redirect_parallel.pl > number running: 2 of 2 > requests sent: 155697 > replies received: 155692 > queue length: 0 > avg service time: 0 msec > > > # FD PID # Requests Flags Time Offset Request > 1 8 21149 104125 > BW 0.033 38 http://s2.xyz.com/1821/78/570/1789/563/i88.js?z=4258 > 81.52.249.106/- - GET myip=10.0.0.165 myport=80\n > 2 9 21150 51572 BW 0.039 0 http://s2.xyz.com/1813/2/570/1781/563/i7.js?z=8853 > 81.52.249.106/- - GET myip=10.0.0.165 myport=80\n > > > Following are my squid settings. > > acl all src 0.0.0.0/0.0.0.0 > acl manager proto cache_object > acl localhost src 127.0.0.1 > acl to_localhost dst 127.0.0.0/255.0.0.0 > acl localnet src 10.0.0.0/255.0.0.0 > acl SSL_ports port 443 > acl Safe_ports port 80 21 443 70 210 1025-65535 280 488 591 777 > acl CONNECT method CONNECT > http_access Allow manager localhost > http_access Deny manager > http_access Deny !Safe_ports > http_access Deny CONNECT !SSL_ports > http_access Allow all > http_access Allow localnet > http_access Deny all > icp_access Allow localnet > icp_access Deny all > htcp_access Allow localnet > htcp_access Deny all > htcp_clr_access Deny all > ident_lookup_access Deny all > http_port 0.0.0.0:80 defaultsite=s1.xyz.com vhost > cache_peer 10.0.0.175 Parent 80 0 no-query round-robin originserver > cache_peer 10.0.0.177 Parent 80 0 no-query round-robin originserver > cache_peer 10.0.0.179 Parent 80 0 no-query round-robin originserver > cache_peer 10.0.0.181 Parent 80 0 no-query round-robin originserver > dead_peer_timeout 10 seconds > hierarchy_stoplist cgi-bin > hierarchy_stoplist ? > cache_mem 0 bytes > maximum_object_size_in_memory 1048576 bytes > memory_replacement_policy lru > cache_replacement_policy lru > cache_dir ufs /home/zdn/squid/var/cache 6000 16 256 IOEngine=Blocking > store_dir_select_algorithm least-load > max_open_disk_fds 0 > minimum_object_size 0 bytes > maximum_object_size 4194304 bytes > cache_swap_low 90 > cache_swap_high 95 > logformat combined %>a %ui %un [%[tl] "%"rm %"ru HTTP/%">v" %Hs %<st > "%"{Referer}>h" "%"{User-Agent}>h" %Ss:%Sh > access_log /home/zdn/squid/var/logs/access.log squid > cache_log /home/zdn/squid/var/logs/cache.log > cache_store_log /home/zdn/squid/var/logs/store.log > logfile_rotate 10 > emulate_httpd_log off > log_ip_on_direct on > mime_table /home/zdn/squid/etc/mime.conf > log_mime_hdrs off > pid_filename /home/zdn/squid/var/logs/squid.pid > debug_options ALL,1 > log_fqdn off > client_netmask 255.255.255.255 > strip_query_terms off > buffered_logs off > url_rewrite_program /home/zdn/bin/redirect_parallel.pl > url_rewrite_children 2 > url_rewrite_concurrency 2000 > url_rewrite_host_header off > url_rewrite_bypass off > refresh_pattern ^ftp: 1440 20% 10080 > > refresh_pattern ^gopher: 1440 0% 1440 > > refresh_pattern (cgi-bin|\?) 0 0% 0 > > refresh_pattern . 0 20% 4320 > > quick_abort_min 16 KB > quick_abort_max 16 KB > quick_abort_pct 95 > read_ahead_gap 16384 bytes > negative_ttl 0 seconds > positive_dns_ttl 21600 seconds > negative_dns_ttl 60 seconds > range_offset_limit 0 bytes > minimum_expiry_time 60 seconds > store_avg_object_size 13 KB > store_objects_per_bucket 20 > request_header_max_size 20480 bytes > reply_header_max_size 20480 bytes > request_body_max_size 0 bytes > via off > ie_refresh off > vary_ignore_expire off > request_entities off > relaxed_header_parser on > forward_timeout 240 seconds > connect_timeout 10 seconds > peer_connect_timeout 5 seconds > read_timeout 120 seconds > request_timeout 10 seconds > persistent_request_timeout 120 seconds > client_lifetime 86400 seconds > half_closed_clients off > pconn_timeout 60 seconds > ident_timeout 10 seconds > shutdown_lifetime 30 seconds > cache_mgr webmaster > mail_program mail > cache_effective_user zdn > httpd_suppress_version_string off > umask 23 > announce_period 31536000 seconds > announce_host tracker.ircache.net > announce_port 3131 > client_persistent_connections off > server_persistent_connections off > persistent_connection_after_error off > detect_broken_pconn off > snmp_port 0 > snmp_access Deny all > snmp_incoming_address 0.0.0.0 > snmp_outgoing_address 255.255.255.255 > icp_port 3130 > htcp_port 0 > log_icp_queries on > udp_incoming_address 0.0.0.0 > udp_outgoing_address 255.255.255.255 > icp_hit_stale off > minimum_direct_hops 4 > minimum_direct_rtt 400 > netdb_low 900 > netdb_high 1000 > netdb_ping_period 300 seconds > query_icmp off > test_reachability off > icp_query_timeout 4000 > maximum_icp_query_timeout 2000 > minimum_icp_query_timeout 5 > background_ping_rate 10 seconds > mcast_icp_query_timeout 2000 > icon_directory /home/zdn/squid/share/icons > global_internal_static on > short_icon_urls on > error_directory /home/zdn/squid/share/errors/templates > err_html_text > email_err_data on > nonhierarchical_direct on > prefer_direct off > incoming_icp_average 6 > incoming_http_average 4 > incoming_dns_average 4 > min_icp_poll_cnt 8 > min_dns_poll_cnt 8 > min_http_poll_cnt 8 > tcp_recv_bufsize 0 bytes > check_hostnames off > allow_underscore on > dns_retransmit_interval 5 seconds > dns_timeout 120 seconds > dns_defnames off > hosts_file /etc/hosts > ignore_unknown_nameservers on > ipcache_size 1024 > ipcache_low 90 > ipcache_high 95 > fqdncache_size 1024 > memory_pools on > memory_pools_limit 5242880 bytes > forwarded_for on > cachemgr_passwd XXXXXXXXXX shutdown config reconfigure offline_toggle > client_db on > refresh_all_ims off > reload_into_ims off > maximum_single_addr_tries 1 > retry_on_error off > as_whois_server whois.ra.net > offline_mode off > uri_whitespace strip > coredump_dir /home/zdn/squid/var/cache > balance_on_multiple_ip on > pipeline_prefetch off > high_response_time_warning 0 > high_page_fault_warning 0 > high_memory_warning 0 bytes > sleep_after_fork 0 > windows_ipaddrchangemonitor on > > Regards > Nitesh > > On Mon, Nov 3, 2008 at 10:27 AM, nitesh naik <niteshnaik@xxxxxxxxx> wrote: >> Henrik / Amos, >> >> Tried using these setting and I could see see delay in serving the >> requests even for cached objects. >> >> 1225687535.330 5459 81.52.249.101 TCP_MEM_HIT/200 1475 GET >> http://abc.xyz.com/3613/172/500/248/211/i5.js?z=9059 - NONE/- >> application/x-javascript >> 1225687535.330 5614 81.52.249.100 TCP_MEM_HIT/200 8129 GET >> http://abc.xyz.com/3357/172/211/4/1/i4.js?z=6079 - NONE/- >> application/x-javascript >> 1225687539.661 12327 168.143.241.12 TCP_MISS/200 2064 GET >> http://bc.xyz.com/2333/254/496/158/122/i17.js?z=3473 - >> ROUNDROBIN_PARENT/10.0.0.181 application/x-javascript >> >> >> Following are timeouts that I have set. >> >> connect_timeout 10 seconds >> peer_connect_timeout 5 seconds >> read_timeout 2 minutes >> request_timeout 10 seconds >> icp_query_timeout 4000 >> >> and cache peer settings. >> >> cache_peer 10.0.0.175 parent 80 0 no-query originserver round-robin >> cache_peer 10.0.0.177 parent 80 0 no-query originserver round-robin >> cache_peer 10.0.0.179 parent 80 0 no-query originserver round-robin >> cache_peer 10.0.0.181 parent 80 0 no-query originserver round-robin >> >> Regards >> Nitesh >> >> >> On Sun, Nov 2, 2008 at 1:11 AM, Henrik Nordstrom >> <henrik@xxxxxxxxxxxxxxxxxxx> wrote: >>> On tor, 2008-10-30 at 19:50 +0530, nitesh naik wrote: >>> >>>> url rewrite helper script works fine for few requests ( 100 req/sec ) >>>> but slows down response as number of requests increase and it takes >>>> 10+ second to deliver the objects. >>> >>> I'v run setups like this at more than thousand requests/s. >>> >>>> Is there way to optimise it further ? >>>> >>>> url_rewrite_program /home/zdn/bin/redirect_parallel.pl >>>> url_rewrite_children 2000 >>>> url_rewrite_concurrency 5 >>> >>> Those two should be the other way around. >>> >>> url_rewrite_concurrency 2000 >>> url_rewrite_children 2 >>> >>> Regards >>> Henrik >>> >> >
