On 26/08/2013 10:49 a.m., glenn.groves wrote:
Hi All, Does anyone have suggestion to fix the error permanently? (I restarted squid which has fixed it right now - but this is not a permanently fix) This morning we had a problem with our squid proxy, it would not accept logons from users, the error in the cache.log was: externalAclLookup: 'memberof' queue overload (ch=0x7f7d873b8358) The above message would repeat about 10 times, then eventually is would seem to authenticate (but still continue to prompt users for a logon – and should not be prompting at all as the users are using Kerberos):
Two things to note here, the authenticate part is done by your auth_param helper(s). The external ACL helper which is overloading is "just" a authorization check based on the already authenticated credentials.
Secondly you have up to 6 lookups sent to the helper per-request. That will fill the helepr queue quite fast. If you can reduce that at all it will help. Otherwise you need to allocate enough helpers that the queue does not overflow.
2013/08/26 07:13:48| externalAclLookup: 'memberof' queue overload (ch=0x7f7d873b8358) 2013/08/26 07:13:48| squid_kerb_auth: DEBUG: ←lots of code→== user@xxxxxxxxxxxxx 2013/08/26 07:13:48| squid_kerb_auth: INFO: User user@xxxxxxxxxxxxx authenticated We only have about 10 users on squid right now and was about to rollout further, I had previously increased the negotiate children to 50 to handle our future 500 users, we are running squid 3.1.10 on centos 6.4, below is the squid.conf:
Maye you should bump the external_acl_type "memberof" helper children up as well to match.
### /etc/squid/squid.conf Configuration File #### ### cache manager cache_mgr helpdesk@xxxxxxxxxxxxx ### negotiate kerberos and ntlm authentication auth_param negotiate program /usr/local/bin/negotiate_wrapper -d --ntlm /usr/bin/ntlm_auth --diagnostics --helper-protocol=squid-2.5-ntlmssp --domain=DOMAIN --kerberos /usr/lib64/squid/squid_kerb_auth -i -d -s HTTP/proxy.domain.com.au auth_param negotiate children 50 auth_param negotiate keep_alive off ### pure ntlm authentication auth_param ntlm program /usr/bin/ntlm_auth --diagnostics --helper-protocol=squid-2.5-ntlmssp --domain=DOMAIN auth_param ntlm children 200 auth_param ntlm keep_alive off ### provide basic authentication via ldap for clients not authenticated via kerberos/ntlm auth_param basic program /usr/lib64/squid/squid_ldap_auth -R -b "dc=domain,dc=com,dc=au" -D squid@xxxxxxxxxxxxx -W /etc/squid/ldappass.txt -f sAMAccountName=%s -h dc1.domain.com.au auth_param basic children 100 auth_param basic realm Internet Proxy auth_param basic credentialsttl 1 minute ### ldap authorisation external_acl_type memberof %LOGIN /usr/lib64/squid/squid_ldap_group -R -K -b "dc=domain,dc=com,dc=au" -D squid@xxxxxxxxxxxxx -W /etc/squid/ldappass.txt -f "(&(objectclass=person)(sAMAccountName=%v)(memberof=cn=%g, ou=Internet,ou=Domain Groups,ou=Domain,dc=domain,dc=com,dc=au))" -h dc1.domain.com.au ### Squid Cache Manager cachemgr_passwd none info cache_dir aufs /var/spool/squid 30000 16 256 minimum_object_size 2 KB maximum_object_size 10 MB cache_swap_low 95 cache_swap_high 97 # aclname acltype typename activedirectorygroup acl SSL method CONNECT acl allowedsites dstdomain "/etc/squid/allowedsites.txt" acl auth proxy_auth REQUIRED acl BlockedAccess external memberof "/etc/squid/blocked_access.txt" acl RestrictedAccess external memberof "/etc/squid/restricted_access.txt" acl StandardAccess external memberof "/etc/squid/standard_access.txt" acl ExceptionAccess external memberof "/etc/squid/exception_access.txt" acl FullAccess external memberof "/etc/squid/full_access.txt" acl AnonymousAccess external memberof "/etc/squid/anonymous_access.txt" acl blockedsites dstdomain "/etc/squid/blockedsites.txt" acl exceptedsites dstdomain "/etc/squid/exceptedsites.txt" acl prioritysites dstdomain "/etc/squid/prioritysites.txt" ### squid defaults acl manager proto cache_object acl localhost src 127.0.0.1/32 ::1 acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1 acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT acl cacheadmin src 192.168.11.221 192.168.8.175 ### http_access rules http_access allow manager localhost http_access allow manager cacheadmin http_access deny manager http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localhost http_access allow prioritysites http_access deny BlockedAccess all http_access allow allowedsites http_access deny RestrictedAccess all http_access allow AnonymousAccess auth http_access allow FullAccess auth http_access allow ExceptionAccess exceptedsites auth
To reduce at least one helepr lookup per request swap this line around to be:
http_access allow exceptedsites ExceptionAccess auth
http_access deny blockedsites http_access allow StandardAccess auth http_access allow auth http_access deny !auth
This "deny !auth" line should be above the "allow AnonymousAccess auth" ... although why you would call it anonymous access while requiring them to also login at the same time (auth ACL test) is beyond me.
http_access deny all ### logging access_log /var/log/squid/access.log squid ### Set memory manually, to allow it to use more of the system cache_mem 1024 MB ### squid defaults http_port 8080 hierarchy_stoplist cgi-bin ? coredump_dir /var/spool/squid refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 Thanks, Glenn
Amos
