On Thu, Nov 12, 2009 at 17:49, Amos Jeffries <squid3@xxxxxxxxxxxxx> wrote: > Kurt Buff wrote: > <snip> >> >>> By testing by running a browser on the proxy server itself you can >>> identify if the problem is Squid or something outside Squid. >> >> Unless you think lynx (or some other text-based browser, or using >> wget/curl/other) is sufficient, I am leery of doing this. I'd have to >> install all the bits to do this, including X and a browser, and all >> that. I'm all up for learning, though, so your suggestion on how to do >> this is very warmly welcomed. > > The squidclient tool fits into the niche gap between telnet and lynx. > Allowing a simple input of the request URL and optional other details and > producing a dump of the results. It has no large dependencies. > > To test through the proxy: > squidclient http://example.com/ > > To test without proxy: > squidclient -h example.com -p 80 / > > >> >> I was unable to do any testing in the past few days - a new child is >> taking me away from working longer hours... >> >> I hope to do some work on it this weekend. >> >> Kurt > > Congratulations! > > Amos Sorry for the very late followup, but I thought I'd outline what happened: For diagnostics, I finally ended up using cURL on three different FreeBSD boxes: One is the router for our DS3 and outside our firewall, and two are internal (used for various network monitoring/management functions) with one of those using our Active Directory DNS servers, and the other using our ISPs DNS servers. This allowed me to test several things at once - inside firewall, outside firewall, internal DNS servers used and external DNS servers used. I selected from the squid logs a set of around 2100 URLs from a single day (basically, any url with 'www' in it), and pared them down to the bare http://FQDN url, and used those as a comparison set across all three machines for multiple runs. Several nights (and days!) I ran the following command at the same time (using at(1)) on each machine: /usr/local/bin/curl -K /root/urls.txt >> /root/DATE-curl-MachineName.out Each entry in urls.txt file looks like the following: url = "http://h30333.www3.hp.com"; -s -w = "%{url_effective}\t%{time_total}\t%{time_namelookup}\n" -o = /dev/null I narrowed it down to that set of options, because I quickly figured out that it wasn't squid at issue, after using it as a proxy in urls.txt with the '-x' option. I had suspected it was the firewall and its name resolution times, which is why I included the '\t%{time_namelookup}' parameter, but that was not the issue. So, to end the suspense - it was the firewall's http(s) proxy, with off-hours results for cURL behind the firewall at over 20 seconds average page load time across the URL set, with some taking as long as 360+ seconds. I did no analysis of which pages were worst, nor why. Applying patches to bring it current for the particular rev of the software fixed the problem completely. It was a weird and frustrating problem, but I feel pretty good about solving it. Kurt
