Amos, Thank you for your help. You're right. Forwarded_for is off in the proxy configuration. When I enable forwarded_for, the remote server does not return that error message. Problem solved. Regards, Dan -----Original Message----- From: Amos Jeffries [mailto:squid3@xxxxxxxxxxxxx] Sent: Wednesday, April 29, 2009 2:02 AM To: Dan Sopher Cc: squid-users@xxxxxxxxxxxxxxx Subject: Re: Proxy/no proxy GET results differ Dan Sopher wrote: > Thank you for the reply. As requested, below are 2 captured TCP streams. The first 3 sections is 1 TCP stream with the proxy in the data path, and captured from each interface of client<->proxy<->remote_server (although can't capture from the remote server interface). The 4th section is the TCP stream without the proxy in the data path. > > Does it appear that the remote server is receiving a request in a format that it cannot process, and is returning the "# Server Error" page to the proxy? > Not really. Apache is normally capable of doing these requests easily. The only non-standard thing about the request going back is the butchered "X-Forwarded-For: unknown" which is unfortunately fairly common, so I doubt its a problem. It's very weird to get a "200 OK" server error. But they would know more about that than any of us. Amos > > > > =============================================================== > PACKET CAPTURE FROM CLIENT WITH SQUID IN DATA PATH (STREAM 1): > > GET http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.1 > TE: deflate,gzip;q=0.3 > Connection: TE, close > Host: mirrors.rpmfusion.org > User-Agent: lwp-request/2.07 > > HTTP/1.0 200 OK > Date: Tue, 28 Apr 2009 17:44:45 GMT > Server: Apache/2.2.11 (Fedora) > Content-Length: 15 > Cache-Control: no-cache > Content-Type: text/plain > X-Cache: MISS from la-squid.twistbox.com > X-Cache-Lookup: MISS from la-squid.twistbox.com:3128 > Via: 1.0 la-squid.twistbox.com (squid) > Proxy-Connection: close > > # Server Error > > =============================================================== > PACKET CAPTURE FROM CLIENT-SIDE TCP PORT OF THE PROXY(STREAM 1): > > GET http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.1 > TE: deflate,gzip;q=0.3 > Connection: TE, close > Host: mirrors.rpmfusion.org > User-Agent: lwp-request/2.07 > > HTTP/1.0 200 OK > Date: Tue, 28 Apr 2009 17:44:45 GMT > Server: Apache/2.2.11 (Fedora) > Content-Length: 15 > Cache-Control: no-cache > Content-Type: text/plain > X-Cache: MISS from la-squid.twistbox.com > X-Cache-Lookup: MISS from la-squid.twistbox.com:3128 > Via: 1.0 la-squid.twistbox.com (squid) > Proxy-Connection: close > > # Server Error > > =============================================================== > PACKET CAPTURE FROM EXTERNAL-SIDE TCP PORT OF THE PROXY(STREAM 1): > > GET /mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.0 > Host: mirrors.rpmfusion.org > User-Agent: lwp-request/2.07 > Via: 1.1 la-squid.twistbox.com (squid) > X-Forwarded-For: unknown > Cache-Control: max-age=259200 > Connection: keep-alive > > HTTP/1.1 200 OK > Date: Tue, 28 Apr 2009 17:44:45 GMT > Server: Apache/2.2.11 (Fedora) > Content-Length: 15 > cache-control: no-cache > Keep-Alive: timeout=2, max=100 > Connection: Keep-Alive > Content-Type: text/plain > > # Server Error > > =============================================================== > PACKET CAPTURE FROM CLIENT _WITHOUT_ THE PROXY IN DATA PATH(STREAM 2): > GET /mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.1 > TE: deflate,gzip;q=0.3 > Connection: TE, close > Host: mirrors.rpmfusion.org > User-Agent: lwp-request/2.07 > > HTTP/1.1 200 OK > Date: Tue, 28 Apr 2009 17:44:49 GMT > Server: Apache/2.2.11 (Fedora) > Content-Length: 404 > cache-control: no-cache > Connection: close > Content-Type: text/plain > > # repo = free-fedora-updates-released-9 arch = i386 country = US > http://mirror.liberty.edu/pub/rpmfusion/free/fedora/updates/9/i386 > http://astromirror.uchicago.edu/rpmfusion/free/fedora/updates/9/i386 > http://mirrors.cat.pdx.edu/rpmfusion/free/fedora/updates/9/i386 > http://mirror.web-ster.com/rpmfusion/free/fedora/updates/9/i386 > http://mirrors.tummy.com/mirrors/rpmfusion.org/free/fedora/updates/9/i386 > > > > > > > -----Original Message----- > From: Amos Jeffries [mailto:squid3@xxxxxxxxxxxxx] > Sent: Monday, April 27, 2009 7:54 PM > To: Dan Sopher > Cc: squid-users@xxxxxxxxxxxxxxx > Subject: Re: Proxy/no proxy GET results differ > >> Hello. >> >> When running the following GET commands from a shell(same results from >> browsers), the results are different, with the error occurring when >> squid(Squid Cache: Version 3.0.STABLE13) is in the data path. Any help is >> appreciated. Thank you in advance. > > You appear to be confused by proper HTTP behavior. The web server responds > in whatever way it needs to for the requests that comes in. > > Squid is not capable of HTTP/1.1 and so sends HTTP/1.0 requests as it is > required to do. > > The only possible thing like a bug I can see here is that squid _might_ be > transforming a TE: encoding request into a Content-Encoding: and the > client unable to handle such encodings despite needing to. We need the > headers input into squid from the client vs the headers sent to Server to > see. > > Amos > >> >> Examples and packet capture data is below: >> >> >> The following are the results WITHOUT squid in the data path: >> >> root@host ~# GET >> "http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-updates-released-9&arch=i386"; >> >> # repo = free-fedora-updates-released-9 arch = i386 country = US >> http://astromirror.uchicago.edu/rpmfusion/free/fedora/updates/9/i386 >> http://mirror.liberty.edu/pub/rpmfusion/free/fedora/updates/9/i386 >> http://mirror.web-ster.com/rpmfusion/free/fedora/updates/9/i386 >> http://mirrors.cat.pdx.edu/rpmfusion/free/fedora/updates/9/i386 >> http://mirrors.tummy.com/mirrors/rpmfusion.org/free/fedora/updates/9/i386 >> >> >> >> and, the following result is from using squid in the data path: >> >> root@host ~# GET -p http://192.168.0.22:3128 >> "http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-updates-released-9&arch=i386"; >> # Server Error >> >> >> >> PACKET CAPTURE DATA: >> =================== >> 1. This is the result without using Squid proxy. Traffic is captured from >> the host using this configuration: >> >> Data path: Host -> Juniper Netscreen firewall -> Router -> Internet >> >> >> GET /mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.1 >> TE: deflate,gzip;q=0.3 >> Connection: TE, close >> Host: mirrors.rpmfusion.org >> User-Agent: lwp-request/2.07 >> >> HTTP/1.1 200 OK >> Date: Fri, 24 Apr 2009 20:53:22 GMT >> Server: Apache/2.2.11 (Fedora) >> Content-Length: 474 >> cache-control: no-cache >> Connection: close >> Content-Type: text/plain >> >> # repo = free-fedora-updates-released-9 arch = i386 country = US >> http://astromirror.uchicago.edu/rpmfusion/free/fedora/updates/9/i386 >> http://mirror.liberty.edu/pub/rpmfusion/free/fedora/updates/9/i386 >> http://mirror.web-ster.com/rpmfusion/free/fedora/updates/9/i386 >> http://mirrors.cat.pdx.edu/rpmfusion/free/fedora/updates/9/i386 >> http://lordmorgul.net/pub/fedora/rpmfusion/free/fedora/updates/9/i386 >> http://mirrors.tummy.com/mirrors/rpmfusion.org/free/fedora/updates/9/i386 >> >> >> >> >> 2. This is the result using Squid proxy. Traffic captured from the >> external interface of the proxy: >> >> Data path: Host -> Squid proxy -> Juniper Netscreen firewall -> Router -> >> Internet >> >> >> GET /mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.0 >> Host: mirrors.rpmfusion.org >> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.9) >> Gecko/2009040821 Firefox/3.0.9 (.NET CLR 3.5.30729) >> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 >> Accept-Language: en-us,en;q=0.5 >> Accept-Encoding: gzip,deflate >> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 >> Keep-Alive: 300 >> Via: 1.0 la-squid.twistbox.com (squid) >> X-Forwarded-For: unknown >> Cache-Control: max-age=259200 >> Connection: keep-alive >> >> HTTP/1.1 200 OK >> Date: Fri, 24 Apr 2009 20:54:51 GMT >> Server: Apache/2.2.9 (Fedora) >> Content-Length: 35 >> content-encoding: gzip >> cache-control: no-cache >> Connection: close >> Content-Type: text/plain; charset=UTF-8 >> >> .....'.I..SV.N-*K-Rp-*./....$}N.... >> >> >> >> CONFIGURATION: >> ============= >> Squid Cache: Version 3.0.STABLE13 >> >> configure options: '--build=i386-redhat-linux-gnu' >> '--host=i386-redhat-linux-gn >> u' '--target=i386-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' >> '--exec- >> prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' >> '--in >> cludedir=/usr/include' '--libdir=/usr/lib' '--libexecdir=/usr/libexec' >> '--shared >> statedir=/usr/com' '--mandir=/usr/share/man' '--infodir=/usr/share/info' >> '--exec >> _prefix=/usr' '--bindir=/usr/sbin' '--libexecdir=/usr/lib/squid' >> '--localstatedi >> r=/var' '--datadir=/usr/share' '--sysconfdir=/etc/squid' >> '--disable-dependency-t >> racking' '--enable-arp-acl' '--enable-auth=basic,digest,ntlm' >> '--enable-basic-au >> th-helpers=LDAP,MSNT,NCSA,PAM,SMB,YP,getpwnam,multi-domain-NTLM,SASL' >> '--enable- >> cache-digests' '--enable-cachemgr-hostname=localhost' >> '--enable-delay-pools' '-- >> enable-digest-auth-helpers=password' '--enable-epoll' >> '--enable-external-acl-hel >> pers=ip_user,ldap_group,unix_group,wbinfo_group' '--enable-icap-client' >> '--enabl >> e-ident-lookups' '--with-large-files' '--enable-linux-netfilter' >> '--enable-ntlm- >> auth-helpers=SMB,fakeauth' '--enable-referer-log' >> '--enable-removal-policies=hea >> p,lru' '--enable-snmp' '--enable-ssl' >> '--enable-storeio=aufs,diskd,null,ufs' '-- >> enable-useragent-log' '--enable-wccpv2' '--with-aio' >> '--with-default-user=squid' >> '--with-filedescriptors=16384' '--with-dl' '--with-openssl=/usr/kerberos' >> '--wi >> th-pthreads' 'build_alias=i386-redhat-linux-gnu' >> 'host_alias=i386-redhat-linux-g >> nu' 'target_alias=i386-redhat-linux-gnu' 'CFLAGS=-fPIE -Os -g -pipe >> -fsigned-cha >> r -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions >> -fstack-protector --pa >> ram=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic >> -fasynchronous-unwind-tabl >> es' 'LDFLAGS=-pie' 'CXXFLAGS=-fPIE -O2 -g -pipe -Wall >> -Wp,-D_FORTIFY_SOURCE=2 -f >> exceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 >> -mtune=g >> eneric -fasynchronous-unwind-tables' 'FFLAGS=-O2 -g -pipe -Wall >> -Wp,-D_FORTIFY_S >> OURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 >> -march=i38 >> 6 -mtune=generic -fasynchronous-unwind-tables' >> >> >> >> >> >> > > -- Please be using Current Stable Squid 2.7.STABLE6 or 3.0.STABLE14 Current Beta Squid 3.1.0.7
