On 2014-01-22 11:44, David Deller wrote:
Hi Amos, thanks for your reply.
On Jan 21, 2014, at 4:29 PM, Amos Jeffries <squid3@xxxxxxxxxxxxx>
wrote:
On 2014-01-22 09:25, David Deller wrote:
Hello,
I’m trying to set up Squid as an HTTPS forward proxy, but I’m having
trouble getting it to work.
Here's some background about my problem:
* I have a web service running on Heroku, with a dynamic IP address.
Static IPs on Heroku are not an option.
* I need to connect to an external web service which is behind a
firewall. The people who operate the external web service will only
open their firewall to a specific static IP.
My attempted solution is to use Squid on a separate server with a
static IP to forward-proxy requests from Heroku to the external
service. That way, the external service always sees the proxy
server's
static IP, instead of the Heroku service's dynamic IP.
Since my proxy server can't rely on an IP address for authentication
You mean authorization, not authentication. A number (IP address)
cannot own credentials, only an entity can have credentials and
credentials are required for authentication.
Sorry - I really do know the difference between the two auths, but it
sometimes comes out wrong.
(that's the problem to begin with!), it must rely on a username and
password. Further, the username and password cannot be transmitted in
clear text,
Meaning: avoid Basic authentication type.
because if an attacker were to intercept that clear text,
then they could connect to my proxy pretending to be me, make
outbound
requests using my proxy's static IP, and thus evade the external web
service's firewall.
Squid is not limited to the Basic authentication protocol. I see you
are already using Digest authentication, which already does what you
want even in unencrypted HTTP.
Therefore, the Squid proxy must only accept connections over HTTPS,
not HTTP. (The connection to the external web service might be HTTP
or
HTTPS.)
No. Given you already use Digest auth the SSL layer is not necessary
to prevent that attack type. It is just an extra layer to obscure the
content being transferred. I think it is a good idea to use it of
possible, but am aware there are some big troubles (namely that modern
browsers still do not support connecting to a proxy over SSL/TLS).
I am distrustful of Digest because it uses MD5. The Wikipedia article
about it seems to suggest that this may not in fact be a problem,
despite MD5’s known weaknesses. Some quick Googling does not suggest a
widespread consensus.
I'm running Squid 3.1.10 on CentOS 6.5.x,
Please try an upgrade. You can find details about newer versions for
CentOS at http://wiki.squid-cache.org/KnowledgeBase/CentOS
I will look into this. I have had problems with third-party
repositories in the past.
<snip>
Using this setup, HTTP proxying works fine, but HTTPS proxying does
not.
Here's an HTTP proxy request from a local box:
$ curl --proxy http://my-proxy-server.example:3128 \
--proxy-anyauth --proxy-user redacted:redacted -w '\n' \
http://urlecho.appspot.com/echo?body=OK
OK
<snip>
Here's another request, this time with HTTPS:
$ curl --proxy https://my-proxy-server.example:3129 \
--proxy-anyauth --proxy-user redacted:redacted -w '\n' \
http://urlecho.appspot.com/echo?body=OK
curl: (56) Recv failure: Connection reset by peer
Nothing in `access.log` after this one, but in `cache.log`:
2014/01/20 20:46:15| clientNegotiateSSL: Error negotiating SSL
connection on FD 10: error:1407609C:SSL
routines:SSL23_GET_CLIENT_HELLO:http request (1/-1)
See the serverfault response. curl is connecting to the proxy using
plain-text instead of SSL.
I want to be able to connect to HTTP servers over the proxy as well,
so changing the protocol of the URL I’m trying to access (per the
serverfault comments) will not help me in cases where the remote web
server does not support HTTPS. Are you saying that Squid does not
support proxying plain HTTP requests over an HTTPS proxy? I could not
find any documentation one way or the other about this.
In any case, I tried it with an HTTPS request to be sure, and it still
failed with the same error (even though the remote web server in this
case does support HTTPS).
This is because your curl test is using plain-TCP to connect to the
proxy. The URL scheme has nothing to do with the curl<->proxy
connection.
Here's the above again, more verbosely:
$ curl -v --proxy https://my-proxy-server.example:3129 \
--proxy-anyauth --proxy-user redacted:redacted -w '\n' \
http://urlecho.appspot.com/echo?body=OK
* Adding handle: conn: 0x7f9a30804000
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7f9a30804000) send_pipe: 1, recv_pipe: 0
* About to connect() to proxy my-proxy-server.example port 3129
(#0)
* Trying proxy.server.IP.redacted...
* Connected to my-proxy-server.example (proxy.server.IP.redacted)
port 3129 (#0)
Note the absence of any mention about SSL/TLS here.
I did notice this and wondered if it might be a problem with curl
itself. So I also tried similar tests with Google Chrome and a Ruby
HTTP library called excon, both of which specifically mention support
of HTTPS proxies. I also tried a few other HTTP libraries that have
HTTP proxy support but don’t specifically mention HTTPS. Since I saw
the same failing result with all of them, I went back to trying to
troubleshoot Squid as the likely source of the problem.
You are using "--proxy https://..."; but the documentation I find for
curl only mentions http:// and various socks*:// proxy schemes supported
for that directive. These tests have all strong signs of curl not
supporting https:// in that parameter.
Also, please be clear on this. The connection to the proxy is *not*
HTTP. It is SSL. With a second-layer HTTP connection happening inside
the SSL. Anything which attempts to send bare/plain HTTP requests to the
proxy https_port will get this same termination and log message. You can
confirm that with telnet:
echo "GET http://example.com/ HTTP/1.0\n\n" | telnet -p 3129
my-proxy-server.example
> GET http://urlecho.appspot.com/echo?body=OK HTTP/1.1
> User-Agent: curl/7.30.0
> Host: urlecho.appspot.com
> Accept: */*
> Proxy-Connection: Keep-Alive
>
* Recv failure: Connection reset by peer
* Closing connection 0
curl: (56) Recv failure: Connection reset by peer
Looks like an SSL error.
It is. Try using --ssl option for curl. You will also need
--proxy-digest to get the Digest auth against the proxy working for
curl.
--proxy-anyauth works fine with HTTP, as in my first example. It
correctly detects that it needs to use Digest auth.
It seems like --ssl refers to the inner HTTP request rather than the
proxy, but the docs are somewhat unclear.
Regardless, I tried with --ssl --proxy-digest just now, and it failed
with the same error.
Okay, that seems to confirm your suspicion about --ssl. It seems that
curl is not capable of doing these tests any more than the web browsers.
However, I'm reusing a subdomain-wildcard SSL
certificate, shown in the above config as `cert.pem` and `key.pem`,
that I've successfully deployed on other web servers. Moreover,
accessing the proxy server directly with curl works, or at least
establishes a connection past the SSL stage:
$ curl https://my-proxy-server.example:3129
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd";>
[--SNIP--]
This tells us that the test machine can access the web serve. Please
run the same test directly on the squid machine. That will tell
whether or not connectivity for Squid is working properly.
There is no web server running on that machine other than Squid. The
request in this example was to Squid directly. Note the (bowdlerized)
hostname and port which match the proxy in the above examples. It was
meant to show that there doesn’t seem to be a problem with the SSL
certificate I have configured with Squid.
You misunderstand. This test is whether curl, running on the same
machine as Squid can perform the same URL request as Squid will be
making on that machine.
Your second test earlier only tested that the web server worked. Not the
connectivity between Squid's machine and that server.
Any ideas what I'm doing wrong? Is what I'm attempting even possible?
It is possible and being done already by some. Although to get the
broken browsers to work properly a VPN/stunnel from the client machine
to proxy port is often needed.
I am not particularly concerned about browsers since I am accessing
the proxy programmatically from a server. I only need an HTTP library
that can talk to Squid correctly.
I take it, then, that you don’t see any obvious problems with my Squid
config for what I’m trying to do?
No, not in the squid config. And the connection is not getting far
enough to identify any cert errors either.
You might actually be better going straight to trying your server app or
whatever will be accessing Squid normally. If it can start opening an
SSL connection to the proxy it is already one step better than your
curl.
Alternatively you could try using stunnel to open a SSL tunnel directly
to the proxy https_port and sending curl requests with that tunnel
IP:port as if it were a plain-HTTP proxy.
Amos
