Hello again :D
I have spent some time debugging my squid.
Yep, it runs as a transparent proxy. The transparent port
is different than 'user' port, but it was not blocked in
any way.
I changed that and also tweaked a few more things. So far
everything works.
Could the problem be caused by hardware failure ? One of
my SATA disks died today, suddenly crashed and it does not
work anymore. Of course, this disk contained SQUID's
cache. Perhaps it was quietly dying for last few days,
however I checked all the logs I could find and didn't
find any complains against disk read/writes.
Btw, is there any way to find how many FDs are in use ? I
have a feeling that my cache does not use more than
100-200 at any time (unless something is wrong). From time
to time SQUID dumps a complain to the cache.log about
request too long or something and it shows FD, it was
never higher than 100 during normal operation.
On Tue, 10 Aug 2010 23:45:09 +1200
Amos Jeffries <squid3@xxxxxxxxxxxxx> wrote:
TJM wrote:
Hello,
For last 3 days I had weird problems with Squid 2.7
stable 9.
I'm running that version since it was released and it
worked just fine
for these few months.
Then suddenly users behind the proxy started to report
serious slowdowns
or downtimes.
This is a very low volume proxy with less than 30 users,
set-up mostly
to save bandwidth during peak hours.
I'm using it myself everyday, so when the last slowdown
started I was
able to look at the logs almost immediately.
When the slowdown event starts, usually the proxy almost
stops
responding - for example, if I open new tab in a brower
and enter URL,
it might take several minutes until it starts loading and
then it might
take another couple of minutes since the page loads
completely (if it
does at all).
It lasts for a while, sometimes it will just go away
without restarting
the proxy, sometimes not.
During the slowdown, any requests made do not appear in
the access log
until the squid handles the request, which as I mentioned
above might
take several minutes.
Also, during last slowdown I've found weird log entries
in the
access.log, a sample from access log, 10.0.0.4 is the
cache IP address:
1281379957.060 899446 10.0.0.4 TCP_MISS/504 227 POST
http://10.0.0.4:3128/p4s - DIRECT/10.0.0.4 text/html
1281379957.060 899446 10.0.0.4 TCP_MISS/504 227 POST
http://10.0.0.4:3128/p4s - DIRECT/10.0.0.4 text/html
1281379957.060 899445 10.0.0.4 TCP_MISS/504 227 POST
http://10.0.0.4:3128/p4s - DIRECT/10.0.0.4 text/html
1281379957.060 899445 10.0.0.4 TCP_MISS/504 227 POST
http://10.0.0.4:3128/p4s - DIRECT/10.0.0.4 text/html
1281379957.060 899445 10.0.0.4 TCP_MISS/504 227 POST
http://10.0.0.4:3128/p4s - DIRECT/10.0.0.4 text/html
Also, the cache.log complains about cache running out of
file
descriptors. Where should I look at to find what's the
cause of this
Yes. That is to be expected if Squid is being forced to
loop requests to itself for extremely long durations.
(Squid holds 3+ FD per request).
problem ? I doubt that it's the config itself, because
the proxy was
running fine for like 7-8 years, upgraded everytime when
stable version
came out.
The strange requests are the proxy machine sending a post
to its own public listening port. Which relays through to
... one guess.
So question is what other POST requests are there that
match that path but don't come from the proxy machine
itself? It's highly likely that client is performing
these requests.
Check your "via" directive is turned on. Something is
permitting these requests to last for over 10 minutes.
The config needs to be corrected to catch and blocking
them quickly. Then monitor the cache.log for loop
warnings to see when it happens.
If you have a transparent proxy check that the port your
firewall passes traffic to is NOT accessible to general
users. Separate ports for the interception and for
regular access are good.
Amos
--
Please be using
Current Stable Squid 2.7.STABLE9 or 3.1.6
Beta testers wanted for 3.2.0.1
