Hi,
I am running squid 3.2.1 on CentOS 6.3 with kerberos authentication
using negotiate_kerberos_auth.
Generally this is working fine, but after some time more and more
helper instances stay busy and cannot finish the given request.
Therefore squid starts new helper processes to have enough working
helpers for kerberos authentication.
This is going on until squid has no more memory for the helpers:
2013/07/30 08:48:04 kid1| Starting new negotiateauthenticator helpers...
2013/07/30 08:48:04 kid1| helperOpenServers: Starting 1/500
'negotiate_kerberos_auth' processes
2013/07/30 08:48:04 kid1| ipcCreate: fork: (12) Cannot allocate memory
2013/07/30 08:48:04 kid1| WARNING: Cannot run
'/usr/lib64/squid/negotiate_kerberos_auth' process.
The problem can only be solved by restarting squid.
squidclient mgr:negotiateauthenticator shows the problem (I put away
the large kerberos requests waiting to be finished):
Negotiate Authenticator Statistics:
program: /usr/lib64/squid/negotiate_kerberos_auth
number active: 39 of 500 (0 shutting down)
requests sent: 11141
replies received: 11133
queue length: 0
avg service time: 4 msec
# FD PID # Requests Flags Time Offset Request
1 19 31373 753 B R 3887.019 0
1 37 31390 755 B R 3637.061 0
1 39 31391 2539 B R 2053.518 0
1 41 31392 78 B R 3859.365 0
1 43 31393 807 B R 2008.036 0
1 57 31396 415 B R 2003.899 0
1 63 31397 363 B R 1975.126 0
1 95 31401 329 B R 1944.980 0
1 29 31491 1891 0.009 0 (none)
1 77 31492 813 0.011 0 (none)
1 88 31493 578 0.009 0 (none)
1 99 31494 430 0.009 0 (none)
1 111 31512 320 0.010 0 (none)
1 115 31513 200 0.018 0 (none)
1 117 31514 158 0.014 0 (none)
1 119 31515 122 0.013 0 (none)
1 121 31516 99 0.011 0 (none)
1 123 31517 82 0.014 0 (none)
1 125 31518 66 0.012 0 (none)
1 118 31519 58 0.010 0 (none)
1 113 32414 44 0.013 0 (none)
1 116 32415 36 0.015 0 (none)
1 124 367 29 0.014 0 (none)
1 128 368 28 0.015 0 (none)
1 137 375 24 0.012 0 (none)
1 138 376 21 0.015 0 (none)
1 140 377 16 0.040 0 (none)
1 142 378 15 0.036 0 (none)
1 144 379 14 0.033 0 (none)
1 139 3490 11 0.037 0 (none)
1 143 3491 10 0.036 0 (none)
1 146 3495 8 0.037 0 (none)
1 148 3496 7 0.046 0 (none)
1 150 3497 6 0.047 0 (none)
1 145 3498 5 0.047 0 (none)
1 149 3499 4 0.041 0 (none)
1 152 3500 3 0.104 0 (none)
1 154 3501 2 0.105 0 (none)
1 156 3502 2 0.089 0 (none)
Flags key:
B = BUSY
C = CLOSING
R = RESERVED
S = SHUTDOWN PENDING
P = PLACEHOLDER
The first eight helper processes are busy and will never return to
normal state until squid is restarted.
Gradually more and more helpers stay in busy state.
strace shows me that this helpers are blocked during a read-command:
....
read(0, "r", 1) = 1
read(0, "r", 1) = 1
read(0, "7", 1) = 1
read(0, "+", 1) = 1
read(0, "a", 1) = 1
read(0, "G", 1) = 1
read(0, <unfinished ...>
After this the process is never continued.
I cannot find any error messages in cache.log even if I switch on
debugging at the helper.
Thank you for help!
Klaus
