Hi All, Issues was with Disk I/O. I have used null cache dir and squid response is much faster now. cache_dir null /empty Thanks everyone for your help. Regards Nitesh On Tue, Nov 4, 2008 at 9:40 AM, nitesh naik <niteshnaik@xxxxxxxxx> wrote: > Does these Redirector statistics mean url rewrite helper program is > slowing down squid response ? avg service time is 1550 msec. > > Redirector Statistics: > program: /home/zdn/bin/redirect_parallel.pl > number running: 2 of 2 > requests sent: 1069753 > replies received: 1069752 > queue length: 0 > avg service time: 1550 msec > > > # FD PID # Requests Flags Time Offset Request > 1 10 18237 12645 B 0.002 38 (none) > 2 15 18238 12335 2.144 0 (none) > > Regards > Nitesh > > On Mon, Nov 3, 2008 at 2:46 PM, nitesh naik <niteshnaik@xxxxxxxxx> wrote: >> Not sure if url rewrite helper is slowing down process because via >> cache manager interface it didn't show any connection back log. What >> information I should look for in cache manager to find out the cause >> of the slow serving of requests ? >> >> Redirector Statistics: >> program: /home/zdn/bin/redirect_parallel.pl >> number running: 2 of 2 >> requests sent: 155697 >> replies received: 155692 >> queue length: 0 >> avg service time: 0 msec >> >> >> # FD PID # Requests Flags Time Offset Request >> 1 8 21149 104125 >> BW 0.033 38 http://s2.xyz.com/1821/78/570/1789/563/i88.js?z=4258 >> 81.52.249.106/- - GET myip=10.0.0.165 myport=80\n >> 2 9 21150 51572 BW 0.039 0 http://s2.xyz.com/1813/2/570/1781/563/i7.js?z=8853 >> 81.52.249.106/- - GET myip=10.0.0.165 myport=80\n >> >> >> Following are my squid settings. >> >> acl all src 0.0.0.0/0.0.0.0 >> acl manager proto cache_object >> acl localhost src 127.0.0.1 >> acl to_localhost dst 127.0.0.0/255.0.0.0 >> acl localnet src 10.0.0.0/255.0.0.0 >> acl SSL_ports port 443 >> acl Safe_ports port 80 21 443 70 210 1025-65535 280 488 591 777 >> acl CONNECT method CONNECT >> http_access Allow manager localhost >> http_access Deny manager >> http_access Deny !Safe_ports >> http_access Deny CONNECT !SSL_ports >> http_access Allow all >> http_access Allow localnet >> http_access Deny all >> icp_access Allow localnet >> icp_access Deny all >> htcp_access Allow localnet >> htcp_access Deny all >> htcp_clr_access Deny all >> ident_lookup_access Deny all >> http_port 0.0.0.0:80 defaultsite=s1.xyz.com vhost >> cache_peer 10.0.0.175 Parent 80 0 no-query round-robin originserver >> cache_peer 10.0.0.177 Parent 80 0 no-query round-robin originserver >> cache_peer 10.0.0.179 Parent 80 0 no-query round-robin originserver >> cache_peer 10.0.0.181 Parent 80 0 no-query round-robin originserver >> dead_peer_timeout 10 seconds >> hierarchy_stoplist cgi-bin >> hierarchy_stoplist ? >> cache_mem 0 bytes >> maximum_object_size_in_memory 1048576 bytes >> memory_replacement_policy lru >> cache_replacement_policy lru >> cache_dir ufs /home/zdn/squid/var/cache 6000 16 256 IOEngine=Blocking >> store_dir_select_algorithm least-load >> max_open_disk_fds 0 >> minimum_object_size 0 bytes >> maximum_object_size 4194304 bytes >> cache_swap_low 90 >> cache_swap_high 95 >> logformat combined %>a %ui %un [%[tl] "%"rm %"ru HTTP/%">v" %Hs %<st >> "%"{Referer}>h" "%"{User-Agent}>h" %Ss:%Sh >> access_log /home/zdn/squid/var/logs/access.log squid >> cache_log /home/zdn/squid/var/logs/cache.log >> cache_store_log /home/zdn/squid/var/logs/store.log >> logfile_rotate 10 >> emulate_httpd_log off >> log_ip_on_direct on >> mime_table /home/zdn/squid/etc/mime.conf >> log_mime_hdrs off >> pid_filename /home/zdn/squid/var/logs/squid.pid >> debug_options ALL,1 >> log_fqdn off >> client_netmask 255.255.255.255 >> strip_query_terms off >> buffered_logs off >> url_rewrite_program /home/zdn/bin/redirect_parallel.pl >> url_rewrite_children 2 >> url_rewrite_concurrency 2000 >> url_rewrite_host_header off >> url_rewrite_bypass off >> refresh_pattern ^ftp: 1440 20% 10080 >> >> refresh_pattern ^gopher: 1440 0% 1440 >> >> refresh_pattern (cgi-bin|\?) 0 0% 0 >> >> refresh_pattern . 0 20% 4320 >> >> quick_abort_min 16 KB >> quick_abort_max 16 KB >> quick_abort_pct 95 >> read_ahead_gap 16384 bytes >> negative_ttl 0 seconds >> positive_dns_ttl 21600 seconds >> negative_dns_ttl 60 seconds >> range_offset_limit 0 bytes >> minimum_expiry_time 60 seconds >> store_avg_object_size 13 KB >> store_objects_per_bucket 20 >> request_header_max_size 20480 bytes >> reply_header_max_size 20480 bytes >> request_body_max_size 0 bytes >> via off >> ie_refresh off >> vary_ignore_expire off >> request_entities off >> relaxed_header_parser on >> forward_timeout 240 seconds >> connect_timeout 10 seconds >> peer_connect_timeout 5 seconds >> read_timeout 120 seconds >> request_timeout 10 seconds >> persistent_request_timeout 120 seconds >> client_lifetime 86400 seconds >> half_closed_clients off >> pconn_timeout 60 seconds >> ident_timeout 10 seconds >> shutdown_lifetime 30 seconds >> cache_mgr webmaster >> mail_program mail >> cache_effective_user zdn >> httpd_suppress_version_string off >> umask 23 >> announce_period 31536000 seconds >> announce_host tracker.ircache.net >> announce_port 3131 >> client_persistent_connections off >> server_persistent_connections off >> persistent_connection_after_error off >> detect_broken_pconn off >> snmp_port 0 >> snmp_access Deny all >> snmp_incoming_address 0.0.0.0 >> snmp_outgoing_address 255.255.255.255 >> icp_port 3130 >> htcp_port 0 >> log_icp_queries on >> udp_incoming_address 0.0.0.0 >> udp_outgoing_address 255.255.255.255 >> icp_hit_stale off >> minimum_direct_hops 4 >> minimum_direct_rtt 400 >> netdb_low 900 >> netdb_high 1000 >> netdb_ping_period 300 seconds >> query_icmp off >> test_reachability off >> icp_query_timeout 4000 >> maximum_icp_query_timeout 2000 >> minimum_icp_query_timeout 5 >> background_ping_rate 10 seconds >> mcast_icp_query_timeout 2000 >> icon_directory /home/zdn/squid/share/icons >> global_internal_static on >> short_icon_urls on >> error_directory /home/zdn/squid/share/errors/templates >> err_html_text >> email_err_data on >> nonhierarchical_direct on >> prefer_direct off >> incoming_icp_average 6 >> incoming_http_average 4 >> incoming_dns_average 4 >> min_icp_poll_cnt 8 >> min_dns_poll_cnt 8 >> min_http_poll_cnt 8 >> tcp_recv_bufsize 0 bytes >> check_hostnames off >> allow_underscore on >> dns_retransmit_interval 5 seconds >> dns_timeout 120 seconds >> dns_defnames off >> hosts_file /etc/hosts >> ignore_unknown_nameservers on >> ipcache_size 1024 >> ipcache_low 90 >> ipcache_high 95 >> fqdncache_size 1024 >> memory_pools on >> memory_pools_limit 5242880 bytes >> forwarded_for on >> cachemgr_passwd XXXXXXXXXX shutdown config reconfigure offline_toggle >> client_db on >> refresh_all_ims off >> reload_into_ims off >> maximum_single_addr_tries 1 >> retry_on_error off >> as_whois_server whois.ra.net >> offline_mode off >> uri_whitespace strip >> coredump_dir /home/zdn/squid/var/cache >> balance_on_multiple_ip on >> pipeline_prefetch off >> high_response_time_warning 0 >> high_page_fault_warning 0 >> high_memory_warning 0 bytes >> sleep_after_fork 0 >> windows_ipaddrchangemonitor on >> >> Regards >> Nitesh >> >> On Mon, Nov 3, 2008 at 10:27 AM, nitesh naik <niteshnaik@xxxxxxxxx> wrote: >>> Henrik / Amos, >>> >>> Tried using these setting and I could see see delay in serving the >>> requests even for cached objects. >>> >>> 1225687535.330 5459 81.52.249.101 TCP_MEM_HIT/200 1475 GET >>> http://abc.xyz.com/3613/172/500/248/211/i5.js?z=9059 - NONE/- >>> application/x-javascript >>> 1225687535.330 5614 81.52.249.100 TCP_MEM_HIT/200 8129 GET >>> http://abc.xyz.com/3357/172/211/4/1/i4.js?z=6079 - NONE/- >>> application/x-javascript >>> 1225687539.661 12327 168.143.241.12 TCP_MISS/200 2064 GET >>> http://bc.xyz.com/2333/254/496/158/122/i17.js?z=3473 - >>> ROUNDROBIN_PARENT/10.0.0.181 application/x-javascript >>> >>> >>> Following are timeouts that I have set. >>> >>> connect_timeout 10 seconds >>> peer_connect_timeout 5 seconds >>> read_timeout 2 minutes >>> request_timeout 10 seconds >>> icp_query_timeout 4000 >>> >>> and cache peer settings. >>> >>> cache_peer 10.0.0.175 parent 80 0 no-query originserver round-robin >>> cache_peer 10.0.0.177 parent 80 0 no-query originserver round-robin >>> cache_peer 10.0.0.179 parent 80 0 no-query originserver round-robin >>> cache_peer 10.0.0.181 parent 80 0 no-query originserver round-robin >>> >>> Regards >>> Nitesh >>> >>> >>> On Sun, Nov 2, 2008 at 1:11 AM, Henrik Nordstrom >>> <henrik@xxxxxxxxxxxxxxxxxxx> wrote: >>>> On tor, 2008-10-30 at 19:50 +0530, nitesh naik wrote: >>>> >>>>> url rewrite helper script works fine for few requests ( 100 req/sec ) >>>>> but slows down response as number of requests increase and it takes >>>>> 10+ second to deliver the objects. >>>> >>>> I'v run setups like this at more than thousand requests/s. >>>> >>>>> Is there way to optimise it further ? >>>>> >>>>> url_rewrite_program /home/zdn/bin/redirect_parallel.pl >>>>> url_rewrite_children 2000 >>>>> url_rewrite_concurrency 5 >>>> >>>> Those two should be the other way around. >>>> >>>> url_rewrite_concurrency 2000 >>>> url_rewrite_children 2 >>>> >>>> Regards >>>> Henrik >>>> >>> >> >
