Hey Guys,
I also have a long term issue with C-ICAP on many servers in different
countries.
It looks like an ICAP Queue overload happening within squid. Since after
taking a lot time in order to fine tune and debug the C-ICAP Service and
the clamav-daemon in order to ensure that it can handle the high load
and is configured properly, the ICAP suspended, down and UP messages
dissappeared entirely in the cache.log, which I had after some days,
then some weeks (after tuning the ICAP Server even more) occuring and
after some messages the performance become steady slow until squid
service is restarted. So now I have fixed the C-ICAP Server that is
using squidclamav as C-ICAP Module for in-stream scanning.
http://squidclamav.darold.net/, So I no longer can see any C-ICAP DOWN
and UP Messages anymore. Which tells me that C-ICAP seems to always have
enough workers to take the load.
So as I said now after finalizing my C-ICAP Server fine tuning,
reconfiguration and debugging. I have no more ICAP Up and down messages
and now after some weeks squid became slow again. When restarting the
squid service all works fine again until some day in some weeks where it
becomes slow again until service restart.
So I guess I need to set the right debug options for you guys, but that
would create a very big log file since it takes days or weeks until this
issue is happening. Any suggestions on what you think how I can do
debugging in the best way?
BTW: I hope I do not mix up issues here, but it seems to be related to
my issue that I am face since I was moving to a newer Version back in
the old days then squid 3.5.12 <- somewhere there this story started for
me. Unsure if it was exactliy starting with this version, some versions
later or some earlier. But well it definitely was not happening on
3.5.10 or earlier, so here I can cut it a bit. So the issue was
definitely introduced somewhere between 3.5.11 until 3.5.18. But as I
said now I can at least fully outline the C-ICAP Server, which the cache
log confirms by not showing any down, suspended and up messages any
more.
I believe when I enforce a service restart once in the night, I will
never see this issue of slow performance caused by the C-ICAP Module
within squid anymore.
Best regards,
Enrico/Flashdown
Am 2017-11-03 12:54, schrieb Eliezer Croitoru:
I'm not sure but after testing with telnet or\and nc you can try to
verify the open files limits on the system which might cause this
issue.
To identify how many connections are opened to the service you can use
netstat or ss tools.
netstat -ntp
and a similar on ss.
Also it's a good practice to put an ICAP periodic(2-20 seconds
apparat) with an OPTIONS request to make sure that the service is
alive.
If you want my testing script let me know.
Eliezer
----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer@xxxxxxxxxxxx
-----Original Message-----
From: squid-users [mailto:squid-users-bounces@xxxxxxxxxxxxxxxxxxxxx]
On Behalf Of Alex Rousskov
Sent: Thursday, November 2, 2017 19:22
To: Stephen Stark <logic4life@xxxxxxxxx>;
squid-users@xxxxxxxxxxxxxxxxxxxxx
Subject: Re: Squid 3.5 ICAP Problems
On 11/02/2017 10:29 AM, Stephen Stark wrote:
Adaptation::Icap::Xaction::noteCommConnected(local=[::]
remote=127.0.0.1:1344 flags=1, errno=101, ...
The logs you have provided do not show where/why exactly the TCP
connection to the ICAP service fails, but error number 101 is probably
"network unreachable". This is unusual but not impossible for
localhost traffic. The next step depends on the failure cause. There
are at least two major cases to consider:
A) If Squid sends packets to 127.0.0.1:1344, then you can easily
reproduce the problem using something like "telnet" or "nc" on the
Squid box command line. Just make sure you use the right _source_ IP
address for the connection! It has to be the same source IP address
that Squid is using. It might not be 127.0.0.1. Running that command
as Squid user might also be important if the Squid box have some fancy
user-specific networking restrictions.
B) If Squid does not send packets to 127.0.0.1:1344, then one can
figure out what goes wrong by studying relevant ALL,9 logs. You may
also want to address other errors or warnings Squid logs to cache.log
(if any).
You can determine whether Squid sends packets to 127.0.0.1:1344 by
collecting a packet trace (for all Squid box interfaces!) and/or
running strace (for the Squid worker process).
HTH,
Alex.
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users
