On 10/01/2013 1:28 a.m., Steve Hill wrote:
On 09/01/13 10:14, Steve Hill wrote:
I have a busy Squid 3.2.3 server that constantly has a huge number of
connections tied up in CLOSE_WAIT (i.e. at the moment it has 364
ESTABLISHED but 3622 in CLOSE_WAIT).
tcp 1 0 ::ffff:172.23.3.254:8080 ::ffff:172.23.2.158:49615
CLOSE_WAIT 32303/(squid-1)
Further to this, it appears that this is triggered by ICAP REQMOD
rewrites of CONNECT requests:
1. Client sends a "CONNECT foo.example.com:443 HTTP/1.1" request to
the proxy.
2. Squid passes the request to the ICAP REQMOD service.
3. The ICAP REQMOD service wants to deny the request, so rewrites the
request.
4. Squid returns a "403 Forbidden" response to the client in clear
text (this is allowed, as it is seen by the client as a response from
the proxy rather than a response from the web server, although very
few clients actually display the page contents these days due to
security restrictions).
5. The client sends a FIN
At this point, the socket stays open on the Squid server - Squid never
closes it and there is 1 byte in the socket's rx queue. I have no
idea what that 1 byte is though - Since all requests are terminated
with a \r\n maybe squid doesn't read the \n ?)
The assumption that all requests are terminated with \r\n is false. The
mime header block is terminated with \r\n and Squid reads that. the
request itself is terminated at the octet specified in the headers
(none, or a specific offset, or a specific chunking code byte).
Does the CONNECT request contain Connection:close or
Connection:keep-alive? Squid supports keep-alive on CONNECT requests in
these situations where the CONNECT size is known and may be waiting for
another client request. Or you may have an orphan connection (bug 3688
side effect or bug 3329 itself).
Please upgrade to 3.2.6 (should anyway for the CVE resolved there) and
see if this issue is gone there.
Amos
