Not sure if url rewrite helper is slowing down process because via cache manager interface it didn't show any connection back log. What information I should look for in cache manager to find out the cause of the slow serving of requests ? Redirector Statistics: program: /home/zdn/bin/redirect_parallel.pl number running: 2 of 2 requests sent: 155697 replies received: 155692 queue length: 0 avg service time: 0 msec # FD PID # Requests Flags Time Offset Request 1 8 21149 104125 BW 0.033 38 http://s2.xyz.com/1821/78/570/1789/563/i88.js?z=4258 81.52.249.106/- - GET myip=10.0.0.165 myport=80\n 2 9 21150 51572 BW 0.039 0 http://s2.xyz.com/1813/2/570/1781/563/i7.js?z=8853 81.52.249.106/- - GET myip=10.0.0.165 myport=80\n Following are my squid settings. acl all src 0.0.0.0/0.0.0.0 acl manager proto cache_object acl localhost src 127.0.0.1 acl to_localhost dst 127.0.0.0/255.0.0.0 acl localnet src 10.0.0.0/255.0.0.0 acl SSL_ports port 443 acl Safe_ports port 80 21 443 70 210 1025-65535 280 488 591 777 acl CONNECT method CONNECT http_access Allow manager localhost http_access Deny manager http_access Deny !Safe_ports http_access Deny CONNECT !SSL_ports http_access Allow all http_access Allow localnet http_access Deny all icp_access Allow localnet icp_access Deny all htcp_access Allow localnet htcp_access Deny all htcp_clr_access Deny all ident_lookup_access Deny all http_port 0.0.0.0:80 defaultsite=s1.xyz.com vhost cache_peer 10.0.0.175 Parent 80 0 no-query round-robin originserver cache_peer 10.0.0.177 Parent 80 0 no-query round-robin originserver cache_peer 10.0.0.179 Parent 80 0 no-query round-robin originserver cache_peer 10.0.0.181 Parent 80 0 no-query round-robin originserver dead_peer_timeout 10 seconds hierarchy_stoplist cgi-bin hierarchy_stoplist ? cache_mem 0 bytes maximum_object_size_in_memory 1048576 bytes memory_replacement_policy lru cache_replacement_policy lru cache_dir ufs /home/zdn/squid/var/cache 6000 16 256 IOEngine=Blocking store_dir_select_algorithm least-load max_open_disk_fds 0 minimum_object_size 0 bytes maximum_object_size 4194304 bytes cache_swap_low 90 cache_swap_high 95 logformat combined %>a %ui %un [%[tl] "%"rm %"ru HTTP/%">v" %Hs %<st "%"{Referer}>h" "%"{User-Agent}>h" %Ss:%Sh access_log /home/zdn/squid/var/logs/access.log squid cache_log /home/zdn/squid/var/logs/cache.log cache_store_log /home/zdn/squid/var/logs/store.log logfile_rotate 10 emulate_httpd_log off log_ip_on_direct on mime_table /home/zdn/squid/etc/mime.conf log_mime_hdrs off pid_filename /home/zdn/squid/var/logs/squid.pid debug_options ALL,1 log_fqdn off client_netmask 255.255.255.255 strip_query_terms off buffered_logs off url_rewrite_program /home/zdn/bin/redirect_parallel.pl url_rewrite_children 2 url_rewrite_concurrency 2000 url_rewrite_host_header off url_rewrite_bypass off refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern (cgi-bin|\?) 0 0% 0 refresh_pattern . 0 20% 4320 quick_abort_min 16 KB quick_abort_max 16 KB quick_abort_pct 95 read_ahead_gap 16384 bytes negative_ttl 0 seconds positive_dns_ttl 21600 seconds negative_dns_ttl 60 seconds range_offset_limit 0 bytes minimum_expiry_time 60 seconds store_avg_object_size 13 KB store_objects_per_bucket 20 request_header_max_size 20480 bytes reply_header_max_size 20480 bytes request_body_max_size 0 bytes via off ie_refresh off vary_ignore_expire off request_entities off relaxed_header_parser on forward_timeout 240 seconds connect_timeout 10 seconds peer_connect_timeout 5 seconds read_timeout 120 seconds request_timeout 10 seconds persistent_request_timeout 120 seconds client_lifetime 86400 seconds half_closed_clients off pconn_timeout 60 seconds ident_timeout 10 seconds shutdown_lifetime 30 seconds cache_mgr webmaster mail_program mail cache_effective_user zdn httpd_suppress_version_string off umask 23 announce_period 31536000 seconds announce_host tracker.ircache.net announce_port 3131 client_persistent_connections off server_persistent_connections off persistent_connection_after_error off detect_broken_pconn off snmp_port 0 snmp_access Deny all snmp_incoming_address 0.0.0.0 snmp_outgoing_address 255.255.255.255 icp_port 3130 htcp_port 0 log_icp_queries on udp_incoming_address 0.0.0.0 udp_outgoing_address 255.255.255.255 icp_hit_stale off minimum_direct_hops 4 minimum_direct_rtt 400 netdb_low 900 netdb_high 1000 netdb_ping_period 300 seconds query_icmp off test_reachability off icp_query_timeout 4000 maximum_icp_query_timeout 2000 minimum_icp_query_timeout 5 background_ping_rate 10 seconds mcast_icp_query_timeout 2000 icon_directory /home/zdn/squid/share/icons global_internal_static on short_icon_urls on error_directory /home/zdn/squid/share/errors/templates err_html_text email_err_data on nonhierarchical_direct on prefer_direct off incoming_icp_average 6 incoming_http_average 4 incoming_dns_average 4 min_icp_poll_cnt 8 min_dns_poll_cnt 8 min_http_poll_cnt 8 tcp_recv_bufsize 0 bytes check_hostnames off allow_underscore on dns_retransmit_interval 5 seconds dns_timeout 120 seconds dns_defnames off hosts_file /etc/hosts ignore_unknown_nameservers on ipcache_size 1024 ipcache_low 90 ipcache_high 95 fqdncache_size 1024 memory_pools on memory_pools_limit 5242880 bytes forwarded_for on cachemgr_passwd XXXXXXXXXX shutdown config reconfigure offline_toggle client_db on refresh_all_ims off reload_into_ims off maximum_single_addr_tries 1 retry_on_error off as_whois_server whois.ra.net offline_mode off uri_whitespace strip coredump_dir /home/zdn/squid/var/cache balance_on_multiple_ip on pipeline_prefetch off high_response_time_warning 0 high_page_fault_warning 0 high_memory_warning 0 bytes sleep_after_fork 0 windows_ipaddrchangemonitor on Regards Nitesh On Mon, Nov 3, 2008 at 10:27 AM, nitesh naik <niteshnaik@xxxxxxxxx> wrote: > Henrik / Amos, > > Tried using these setting and I could see see delay in serving the > requests even for cached objects. > > 1225687535.330 5459 81.52.249.101 TCP_MEM_HIT/200 1475 GET > http://abc.xyz.com/3613/172/500/248/211/i5.js?z=9059 - NONE/- > application/x-javascript > 1225687535.330 5614 81.52.249.100 TCP_MEM_HIT/200 8129 GET > http://abc.xyz.com/3357/172/211/4/1/i4.js?z=6079 - NONE/- > application/x-javascript > 1225687539.661 12327 168.143.241.12 TCP_MISS/200 2064 GET > http://bc.xyz.com/2333/254/496/158/122/i17.js?z=3473 - > ROUNDROBIN_PARENT/10.0.0.181 application/x-javascript > > > Following are timeouts that I have set. > > connect_timeout 10 seconds > peer_connect_timeout 5 seconds > read_timeout 2 minutes > request_timeout 10 seconds > icp_query_timeout 4000 > > and cache peer settings. > > cache_peer 10.0.0.175 parent 80 0 no-query originserver round-robin > cache_peer 10.0.0.177 parent 80 0 no-query originserver round-robin > cache_peer 10.0.0.179 parent 80 0 no-query originserver round-robin > cache_peer 10.0.0.181 parent 80 0 no-query originserver round-robin > > Regards > Nitesh > > > On Sun, Nov 2, 2008 at 1:11 AM, Henrik Nordstrom > <henrik@xxxxxxxxxxxxxxxxxxx> wrote: >> On tor, 2008-10-30 at 19:50 +0530, nitesh naik wrote: >> >>> url rewrite helper script works fine for few requests ( 100 req/sec ) >>> but slows down response as number of requests increase and it takes >>> 10+ second to deliver the objects. >> >> I'v run setups like this at more than thousand requests/s. >> >>> Is there way to optimise it further ? >>> >>> url_rewrite_program /home/zdn/bin/redirect_parallel.pl >>> url_rewrite_children 2000 >>> url_rewrite_concurrency 5 >> >> Those two should be the other way around. >> >> url_rewrite_concurrency 2000 >> url_rewrite_children 2 >> >> Regards >> Henrik >> >
