----- "Amos Jeffries" <squid3@xxxxxxxxxxxxx> wrote: > On Wed, 3 Feb 2010 20:14:36 -0800 (GMT-08:00), Shawn Wright > <swright@xxxxxxxxxxxx> wrote: > > On Monday, our usually very stable Squid 2.6stable20 server acted up, > due > > to what appears to be a problem with the ntlm_auth helpers. No changes > have > > been made to the system recently, with the exception of recompiling to > > allow for logs >2Gb last month. Prior to that, the system had been > stable > > since Mar 2008. The logs below show a FATAL stop after too many queued > > auths, which could be due to slow AD DCs combined with spyware on > clients > > hammering the proxy. But the odd thing is after the restart at 19:36 on > Feb > > 1, our MRTG snmp stats showed a gradual decline in active clients from a > > normal 650 down to 0, over a period of 24 hours. All other snmp stats > > appear normal. > > > > These two events appear to be related, but I'm not sure how or where to > > look next. > > Do the logs over that 24 hr period confirm the loss of client connections? > They should at least have clues as to what the last action each client did > was. The final interaction between a client and the proxy might be > significant if it was repeated many times across the client base. That's the odd part. After the restart, all activity appeared to continue as normal - stats showed 25mbps traffic and ~100rps as we'd normally see, and logs indicated lots of authenticated users (nearly all access except windows updates, etc. require auth). The only clue that a problem exists is the zero counter on the snmp data for active clients. > Bumping the number of helpers up a bit will raise the ceiling on that > happening again. I'll give that a try, and also look at our DCs to see if any clues lie there. > This is just Squid axing away at the helpers. They were actually busy > doing what they are supposed to do without any problem, so they complain > when aborted. Not their fault squid did not have enough running to cope > with the load. Thanks for the info!