On 8/04/2013 4:24 a.m., Christopher H. Laco wrote:
Ok, I've solved this as much as I need too without digging into the squid source itself. I fired up tcpdump and took a capture of the failed attempt from the proxy to the keyserver using 3.1.19, then a capture of the successful attempt from the proxy to the keyserver using 3.3.3 (sorry for the dots, pulled directly from tcpdump -XX) Failed: GET./.HTTP/1.1.. User-Agent:.curl/7.22.0.(86_64-pc-linux-gnu).libcurl/7.22.0.OpenSSL/1.0.1.zlib/1.2.3.4.libidn/1.23.librtmp/2.3.. Host:.keyserver.ubuntu.com.. Accept:.*/*.. Via:.1.1.localhost.(squid/3.1.19).. X-Forwarded-For:.10.10.10.20. Cache-Control:.max-age=259200.. Connection:.keep-alive.... Success: GET./.HTTP/1.1.. User-Agent:.curl/7.22.0.(x86_64-pc-linux-gnu).libcurl/7.22.0.OpenSSL/1.0.1.zlib/1.2.3.4.libidn/1.23.librtmp/2.3.. Host:.keyserver.ubuntu.com.. Accept:.*/*.. Via:.1.1.opencenter-proxy.(squid/3.3.3).. X-Forwarded-For:.10.10.10.20.. Cache-Control:.max-age=259200.. Connection:.keep-alive.... The only difference in the request is the hostname/version in Via The difference in the response is that for the Failure, the remote keyserver is actually the one returning the Squid Access Denied page. That wasn't apparent since my local proxy default is also "localhost". 14:59:10.690245 IP cassava.canonical.com.http > 10.0.2.15.52015: Flags [P.], seq 1:1389, ack 293, win 65535, length 1388 0x0000: 0800 2788 0ca6 5254 0012 3502 0800 4500 ..'...RT..5...E. 0x0010: 0594 6563 0000 4006 4dfe 5bbd 5a37 0a00 ..ec..@.M.[.Z7.. 0x0020: 020f 0050 cb2f 1205 4802 3a61 994a 5018 ...P./..H.:a.JP. 0x0030: ffff 0054 0000 4854 5450 2f31 2e30 2034 ...T..HTTP/1.0.4 0x0040: 3033 2046 6f72 6269 6464 656e 0d0a 5365 03.Forbidden..Se 0x0050: 7276 6572 3a20 7371 7569 642f 332e 312e rver:.squid/3.1. 0x0060: 3139 0d0a 4d69 6d65 2d56 6572 7369 6f6e 19..Mime-Version 0x0070: 3a20 312e 300d 0a44 6174 653a 2053 756e :.1.0..Date:.Sun 0x0080: 2c20 3037 2041 7072 2032 3031 3320 3134 ,.07.Apr.2013.14 0x0090: 3a35 393a 3130 2047 4d54 0d0a 436f 6e74 :59:10.GMT..Cont 0x00a0: 656e 742d 5479 7065 3a20 7465 7874 2f68 ent-Type:.text/h 0x00b0: 746d 6c0d 0a43 6f6e 7465 6e74 2d4c 656e tml..Content-Len 0x00c0: 6774 683a 2033 3430 380d 0a58 2d53 7175 gth:.3408..X-Squ 0x00d0: 6964 2d45 7272 6f72 3a20 4552 525f 4143 id-Error:.ERR_AC 0x00e0: 4345 5353 5f44 454e 4945 4420 300d 0a56 CESS_DENIED.0..V .... At this point, I changed visible_hostname to something other than localhost, and now the requests through 3.1.19 work as expected. So, with that said, it seems that this problem is some form of logic issue within the 3.1.19 codebase where if the source and upstream proxies are named the same (and the same version*), shenanigans ensue with the upstream proxy response. Maybe this is a setting somewhere I don't know about or some form of circular processing protection gone wrong.
What you are looking at is the feature we call "forwarding loop detection". The Via: header is scanned by each proxy for entries of *its own* (supposedly unique) combination of HTTP protocol support level, machine hostname (unique all by itself when one follows the RFCs), proxy software version, and some extra fluff header octets to prevent false matches.
As you found when all three of these details are exactly lining up ... including the setting of your proxies hostname to that of the remote upstream proxy (uhm, "localhost"). The upstream *will* reject your traffic to protect itself against abuse.
NP: For anyone reading this. If you are an admin needing to share visible_hostname values between multiple proxies in a hierarchy look at the unique_hostname directive.
*Now, even more interesting, if I change visible_hostname to localhost in the 3.3.3 version and make the request to the upstream keyserver, things also work. Maybe this is just an issue with the 3.1.x codebase as it was. I might test this local with a few test proxy servers and various visible_hostname incantations. To recap: - 3.1.19 visible_hostname defaults to localhost. always change it. :-)
Well, no and maybe... it defaults to localhost only if gethostname() fails along with an rDNS lookup on the resulting name. This is true for all Squid right back to Squid-1.1 AFAIK. What differs is the amount of bugginess in the gethostname() and rDNS lookup logics - 3.1 had a few intolerances which made localhost appear more often than later releases.
- 3.3.3 visible_hostname defaults to gethosename() - 3.1.19 connecting to an upstream 3.1.19, both with the same name "localhost" returns access denied. - keyserver.ubuntu.com should probably fix it's visible_hostname setting.
Sadly they are *far* from alone in doing so ... Amos
