Search squid archive

Re: CLOSE_WAIT

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 11/01/2013 7:11 a.m., Steve Hill wrote:
On 09/01/13 21:07, Amos Jeffries wrote:

Does the CONNECT request contain Connection:close or
Connection:keep-alive? Squid supports keep-alive on CONNECT requests in
these situations where the CONNECT size is known and may be waiting for
another client request.

The client sends "Proxy-Connection: keep-alive", so it would indeed be possible for the connection to be reused. I should add that I'm seeing this with a non-transparent configuration (i.e. without Tproxy and without ssl_bump).

Please upgrade to 3.2.6 (should anyway for the CVE resolved there) and
see if this issue is gone there.

I've now upgraded my test server and the problem remains.

The "403 Forbidden" being sent back to the client didn't have a Content Length header, which of course caused the client to drop the connection. If I add a Content Length, the connection stays alive and I think it's a bit more obvious whats happening:

1. Client connects to Squid and sends a CONNECT.
2. Squid passes the request to the ICAP REQMOD service.
3. ICAP rewrites the request.
4. Squid returns a "403 Forbidden" to the client in the clear. At this point, the connection is still alive with empty queues. Squid's cache.log shows: 2013/01/10 17:52:18 kid1| abandoning local=[2a00:1a90:5::9]:3128 remote=[2001:4d48:ad51:501:226:bbff:fe18:f3ff]:49926 FD 19 flags=1

Now, the user retries the connection:
5. Client sends CONNECT over the existing connection.
6. Squid does not read the socket (netstat shows the rx queue on the socket is 220 octets long, which is the whole request the client just sent).
7. The client sits there spinning.
8. Eventually the client times out and drops the connection.
The connection is now in CLOSE_WAIT on the Squid server and will remain like that indefinitely.

So it seems apparent that after Squid delivers the clear-text response, it abandons the socket but never closes it. From looking in the source, this is client_side.cc, and it has a comment:
    // XXX: Can this happen? CONNECT tunnels have deferredRequest set.
It looks to me as if the (conn->flags.readMore) section above should be the bit being executed, although I don't quite understand deferred requests. In either case, it seems like we should close the socket if it ever gets abandoned?


A few notes on the REQMOD rewrite:
My ICAP service can generate a "403 Forbidden" response during REQMOD, or rewrite the request to a GET to a local web server. In the latter case, the local webserver generates a "403 Forbidden" response. In both cases the client sees a similar response to its request, and in both cases this bug manifests.

Interestingly, if the ICAP service closes the ICAP connection instead of returning a response, Squid generates a "500 Internal Server Error" and does not abandon the socket (the client then drops the connection, which squid handles correctly, and therefore doesn't end in CLOSE_WAIT).


Okay. So the source of the problem is that Squid thinks there is something using the socket, but it never got into the tunnel code which would have closed it?
 Is the 403 message being generated inside Squid or by ICAP?


In the case where the server generates the 403 and things hang it will be because Squid is expecting a close to arrive from server or client. Once a tunnel is open and transmitting data it is up to those endpoints to ensure keep-alive and other such details, although Squid applies a timeout since last byte transferred.

Amos


[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux