I am using Squid 3.1.0.14, configured to make REQMOD and RESPMOD requests to a local ICAP server. Everything seems to work fine, except the number of connections between Squid and the ICAP server sometimes keeps increasing over the course of days or weeks.
I haven't been able to figure out what is triggering the problem, but it appears that in certain circumstances, Squid just stops using one of the ICAP connections - from what I can tell, the ICAP server has finished dealing with a request and is waiting for the next one, but Squid never sends a new request. Squid continues to operate ok, bringing up more ICAP connections to accommodate more client requests whilst the "lost" connection stays dormant.
The ICAP server is configured to allow a maximum of 100 concurrent connections and eventually, the number of lost connections becomes so great that this limit is reached and the ICAP server starts rejecting the new connections that Squid is bringing up. At this point the users start getting Squid's ICAP error page.
Since I'm unfamiliar with the internal structure of Squid, I'm not really sure where to begin with debugging Squid itself. I think I've done as much debugging as is possible from the ICAP server side (this seems to indicate that the ICAP session itself has been fine - the last ICAP request from Squid looks fine and has terminated and the last ICAP response from the ICAP server looks fine and the server is sat waiting for a new request that never comes).
This problem isn't something that can be reliably worked around on the ICAP server end - the ICAP server has no way of knowing if a connection from Squid has been "lost" (i.e. still open but will never again be used) or if it is simply idle. Because of this, having the ICAP server time out idle connections would introduce a race condition - if the connection is just idle, rather than lost, the ICAP server might time it out and close it just as Squid starts sending a new request; in this case the request would fail and the user would get an error page.
Any suggestions on how to debug the problem would be greatfully received. Thanks. -- - Steve Hill Technical Director Opendium Limited http://www.opendium.com
