Dan Sopher wrote:
Thank you for the reply. As requested, below are 2 captured TCP streams. The first 3 sections is 1 TCP stream with the proxy in the data path, and captured from each interface of client<->proxy<->remote_server (although can't capture from the remote server interface). The 4th section is the TCP stream without the proxy in the data path.
Does it appear that the remote server is receiving a request in a format that it cannot process, and is returning the "# Server Error" page to the proxy?
Not really. Apache is normally capable of doing these requests easily.
The only non-standard thing about the request going back is the
butchered "X-Forwarded-For: unknown" which is unfortunately fairly
common, so I doubt its a problem.
It's very weird to get a "200 OK" server error. But they would know more
about that than any of us.
Amos
===============================================================
PACKET CAPTURE FROM CLIENT WITH SQUID IN DATA PATH (STREAM 1):
GET http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: mirrors.rpmfusion.org
User-Agent: lwp-request/2.07
HTTP/1.0 200 OK
Date: Tue, 28 Apr 2009 17:44:45 GMT
Server: Apache/2.2.11 (Fedora)
Content-Length: 15
Cache-Control: no-cache
Content-Type: text/plain
X-Cache: MISS from la-squid.twistbox.com
X-Cache-Lookup: MISS from la-squid.twistbox.com:3128
Via: 1.0 la-squid.twistbox.com (squid)
Proxy-Connection: close
# Server Error
===============================================================
PACKET CAPTURE FROM CLIENT-SIDE TCP PORT OF THE PROXY(STREAM 1):
GET http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: mirrors.rpmfusion.org
User-Agent: lwp-request/2.07
HTTP/1.0 200 OK
Date: Tue, 28 Apr 2009 17:44:45 GMT
Server: Apache/2.2.11 (Fedora)
Content-Length: 15
Cache-Control: no-cache
Content-Type: text/plain
X-Cache: MISS from la-squid.twistbox.com
X-Cache-Lookup: MISS from la-squid.twistbox.com:3128
Via: 1.0 la-squid.twistbox.com (squid)
Proxy-Connection: close
# Server Error
===============================================================
PACKET CAPTURE FROM EXTERNAL-SIDE TCP PORT OF THE PROXY(STREAM 1):
GET /mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.0
Host: mirrors.rpmfusion.org
User-Agent: lwp-request/2.07
Via: 1.1 la-squid.twistbox.com (squid)
X-Forwarded-For: unknown
Cache-Control: max-age=259200
Connection: keep-alive
HTTP/1.1 200 OK
Date: Tue, 28 Apr 2009 17:44:45 GMT
Server: Apache/2.2.11 (Fedora)
Content-Length: 15
cache-control: no-cache
Keep-Alive: timeout=2, max=100
Connection: Keep-Alive
Content-Type: text/plain
# Server Error
===============================================================
PACKET CAPTURE FROM CLIENT _WITHOUT_ THE PROXY IN DATA PATH(STREAM 2):
GET /mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: mirrors.rpmfusion.org
User-Agent: lwp-request/2.07
HTTP/1.1 200 OK
Date: Tue, 28 Apr 2009 17:44:49 GMT
Server: Apache/2.2.11 (Fedora)
Content-Length: 404
cache-control: no-cache
Connection: close
Content-Type: text/plain
# repo = free-fedora-updates-released-9 arch = i386 country = US
http://mirror.liberty.edu/pub/rpmfusion/free/fedora/updates/9/i386
http://astromirror.uchicago.edu/rpmfusion/free/fedora/updates/9/i386
http://mirrors.cat.pdx.edu/rpmfusion/free/fedora/updates/9/i386
http://mirror.web-ster.com/rpmfusion/free/fedora/updates/9/i386
http://mirrors.tummy.com/mirrors/rpmfusion.org/free/fedora/updates/9/i386
-----Original Message-----
From: Amos Jeffries [mailto:squid3@xxxxxxxxxxxxx]
Sent: Monday, April 27, 2009 7:54 PM
To: Dan Sopher
Cc: squid-users@xxxxxxxxxxxxxxx
Subject: Re: Proxy/no proxy GET results differ
Hello.
When running the following GET commands from a shell(same results from
browsers), the results are different, with the error occurring when
squid(Squid Cache: Version 3.0.STABLE13) is in the data path. Any help is
appreciated. Thank you in advance.
You appear to be confused by proper HTTP behavior. The web server responds
in whatever way it needs to for the requests that comes in.
Squid is not capable of HTTP/1.1 and so sends HTTP/1.0 requests as it is
required to do.
The only possible thing like a bug I can see here is that squid _might_ be
transforming a TE: encoding request into a Content-Encoding: and the
client unable to handle such encodings despite needing to. We need the
headers input into squid from the client vs the headers sent to Server to
see.
Amos
Examples and packet capture data is below:
The following are the results WITHOUT squid in the data path:
root@host ~# GET
"http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-updates-released-9&arch=i386";
# repo = free-fedora-updates-released-9 arch = i386 country = US
http://astromirror.uchicago.edu/rpmfusion/free/fedora/updates/9/i386
http://mirror.liberty.edu/pub/rpmfusion/free/fedora/updates/9/i386
http://mirror.web-ster.com/rpmfusion/free/fedora/updates/9/i386
http://mirrors.cat.pdx.edu/rpmfusion/free/fedora/updates/9/i386
http://mirrors.tummy.com/mirrors/rpmfusion.org/free/fedora/updates/9/i386
and, the following result is from using squid in the data path:
root@host ~# GET -p http://192.168.0.22:3128
"http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-updates-released-9&arch=i386";
# Server Error
PACKET CAPTURE DATA:
===================
1. This is the result without using Squid proxy. Traffic is captured from
the host using this configuration:
Data path: Host -> Juniper Netscreen firewall -> Router -> Internet
GET /mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: mirrors.rpmfusion.org
User-Agent: lwp-request/2.07
HTTP/1.1 200 OK
Date: Fri, 24 Apr 2009 20:53:22 GMT
Server: Apache/2.2.11 (Fedora)
Content-Length: 474
cache-control: no-cache
Connection: close
Content-Type: text/plain
# repo = free-fedora-updates-released-9 arch = i386 country = US
http://astromirror.uchicago.edu/rpmfusion/free/fedora/updates/9/i386
http://mirror.liberty.edu/pub/rpmfusion/free/fedora/updates/9/i386
http://mirror.web-ster.com/rpmfusion/free/fedora/updates/9/i386
http://mirrors.cat.pdx.edu/rpmfusion/free/fedora/updates/9/i386
http://lordmorgul.net/pub/fedora/rpmfusion/free/fedora/updates/9/i386
http://mirrors.tummy.com/mirrors/rpmfusion.org/free/fedora/updates/9/i386
2. This is the result using Squid proxy. Traffic captured from the
external interface of the proxy:
Data path: Host -> Squid proxy -> Juniper Netscreen firewall -> Router ->
Internet
GET /mirrorlist?repo=free-fedora-updates-released-9&arch=i386 HTTP/1.0
Host: mirrors.rpmfusion.org
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.9)
Gecko/2009040821 Firefox/3.0.9 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Via: 1.0 la-squid.twistbox.com (squid)
X-Forwarded-For: unknown
Cache-Control: max-age=259200
Connection: keep-alive
HTTP/1.1 200 OK
Date: Fri, 24 Apr 2009 20:54:51 GMT
Server: Apache/2.2.9 (Fedora)
Content-Length: 35
content-encoding: gzip
cache-control: no-cache
Connection: close
Content-Type: text/plain; charset=UTF-8
.....'.I..SV.N-*K-Rp-*./....$}N....
CONFIGURATION:
=============
Squid Cache: Version 3.0.STABLE13
configure options: '--build=i386-redhat-linux-gnu'
'--host=i386-redhat-linux-gn
u' '--target=i386-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr'
'--exec-
prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc'
'--in
cludedir=/usr/include' '--libdir=/usr/lib' '--libexecdir=/usr/libexec'
'--shared
statedir=/usr/com' '--mandir=/usr/share/man' '--infodir=/usr/share/info'
'--exec
_prefix=/usr' '--bindir=/usr/sbin' '--libexecdir=/usr/lib/squid'
'--localstatedi
r=/var' '--datadir=/usr/share' '--sysconfdir=/etc/squid'
'--disable-dependency-t
racking' '--enable-arp-acl' '--enable-auth=basic,digest,ntlm'
'--enable-basic-au
th-helpers=LDAP,MSNT,NCSA,PAM,SMB,YP,getpwnam,multi-domain-NTLM,SASL'
'--enable-
cache-digests' '--enable-cachemgr-hostname=localhost'
'--enable-delay-pools' '--
enable-digest-auth-helpers=password' '--enable-epoll'
'--enable-external-acl-hel
pers=ip_user,ldap_group,unix_group,wbinfo_group' '--enable-icap-client'
'--enabl
e-ident-lookups' '--with-large-files' '--enable-linux-netfilter'
'--enable-ntlm-
auth-helpers=SMB,fakeauth' '--enable-referer-log'
'--enable-removal-policies=hea
p,lru' '--enable-snmp' '--enable-ssl'
'--enable-storeio=aufs,diskd,null,ufs' '--
enable-useragent-log' '--enable-wccpv2' '--with-aio'
'--with-default-user=squid'
'--with-filedescriptors=16384' '--with-dl' '--with-openssl=/usr/kerberos'
'--wi
th-pthreads' 'build_alias=i386-redhat-linux-gnu'
'host_alias=i386-redhat-linux-g
nu' 'target_alias=i386-redhat-linux-gnu' 'CFLAGS=-fPIE -Os -g -pipe
-fsigned-cha
r -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector --pa
ram=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic
-fasynchronous-unwind-tabl
es' 'LDFLAGS=-pie' 'CXXFLAGS=-fPIE -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -f
exceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386
-mtune=g
eneric -fasynchronous-unwind-tables' 'FFLAGS=-O2 -g -pipe -Wall
-Wp,-D_FORTIFY_S
OURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32
-march=i38
6 -mtune=generic -fasynchronous-unwind-tables'
--
Please be using
Current Stable Squid 2.7.STABLE6 or 3.0.STABLE14
Current Beta Squid 3.1.0.7
