On 20/04/2015 7:31 p.m., Jagannath Naidu wrote: > Hi, > > I am having this issue very frequently. Please help on this. > > I get these errors randomly, mostly when usage is at very peak. (800 users) > > > /var/log/squid/cache.log > > 2015/04/20 12:37:40| externalAclLookup: 'wbinfo_group_helper' queue > overload (ch=0x7fc99e2ce518) What do you think "overload" means? The helper is unable to cope with the traffic load being passed to it. Here is the biggest hint: > > in /var/log/messages, I get the following errors > > pr 20 12:59:15 GGNPROXY01 winbindd[1910]: winbindd: Exceeding 200 client > connections, no idle connection found > Then squid stops working. For squid to start work again, I have to dlete > the cache and restart the squid "squid -k reconfigure", and then squid > restart. What Squid version are you using? > > squid.conf > > max_filedesc 17192 > acl manager proto cache_object > acl localhost src 172.16.50.61/24 You have an entire /24 (256 IPs) assigned to this machine? I think you need to remove that "/24" part if the *.61 is the local machines *public* IP. > http_access allow manager localhost > dns_nameservers 172.16.3.34 10.1.2.91 > acl allowips src 172.16.58.187 172.16.16.192 172.16.58.113 172.16.58.63 > 172.16.58.98 172.16.60.244 172.16.58.165 172.16.58.157 > http_access allow allowips > auth_param basic realm Squid proxy-caching web server > auth_param basic credentialsttl 2 hours external_acl_type nt_group ttl=0 > children=60 %LOGIN /usr/lib64/squid/wbinfo_group.pl The above two very mangled config lines are useless. Remove them. > acl localnet src 172.16.0.0/24 Its a bit strange that none of the localhost machine IPs (172.16.50.0-172.16.50.255) are part of the LAN its plugged into 172.16.0.0-172.16.0.255. > acl localnet src fc00::/7 # RFC 4193 local private network range > acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines > auth_param ntlm program /usr/bin/ntlm_auth --diagnostics > --helper-protocol=squid-2.5-ntlmssp --domain=HTMEDIA.NET Okay you have configured NTLM... > auth_param ntlm program /usr/bin/ntlm_auth > --helper-protocol=squid-2.5-ntlmssp --domain=HTMEDIA.NET ... but twice. With different settings. Only these last ones will have any effect. > auth_param ntlm children 600 > auth_param ntlm keep_alive off > auth_param negotiate children 150 > auth_param negotiate keep_alive off > visible_hostname GGNPROXY01.HTMEDIA.NET > external_acl_type wbinfo_group_helper ttl=0 children=40 %LOGIN > /usr/lib64/squid/wbinfo_group.pl -d > auth_param negotiate keep_alive off You have several useless configuration lines for Negotiate auth which is not being used in any way. Remove those. > acl Safe_ports port 8080 #https > acl SSL_ports port 443 > acl Safe_ports port 80 # http > acl Safe_ports port 21 # ftp > acl Safe_ports port 443 # https > acl Safe_ports port 70 # gopher > acl Safe_ports port 210 # wais > acl Safe_ports port 1025-65535 # unregistered ports > acl Safe_ports port 280 # http-mgmt > acl Safe_ports port 488 # gss-http > acl Safe_ports port 591 # filemaker > acl Safe_ports port 777 # multiling http > acl CONNECT method CONNECT > acl auth proxy_auth REQUIRED > acl google dstdomain -i "/etc/squid/google_site.com" > http_access allow google > acl sq1 external wbinfo_group_helper "/etc/squid/HT/sq1" > acl sq2 external wbinfo_group_helper "/etc/squid/HT/sq2" > acl sq3 external wbinfo_group_helper "/etc/squid/HT/sq3" > acl sq4 external wbinfo_group_helper "/etc/squid/HT/sq4" > acl sq5 external wbinfo_group_helper "/etc/squid/HT/sq5" > acl pro1 external wbinfo_group_helper "/etc/squid/HT/pro1" > acl pro2 external wbinfo_group_helper "/etc/squid/HT/pro2" > acl pro3 external wbinfo_group_helper "/etc/squid/HT/pro3" > acl pro4 external wbinfo_group_helper "/etc/squid/HT/pro4" > acl pro5 external wbinfo_group_helper "/etc/squid/HT/pro5" > acl pro6 external wbinfo_group_helper "/etc/squid/HT/pro6" > acl webvip external wbinfo_group_helper "/etc/squid/HT/webvip" > acl allgroup external wbinfo_group_helper "/etc/squid/HT/allgreop" > acl restricted external wbinfo_group_helper "/etc/squid/HT/restricted" > acl ad_auth proxy_auth REQUIRE You already have an ACL named "auth" which performs authentication. The above line is not useful. Remove it and replace all uses of "ad_auth" ACL with "auth" ACL. > acl allowwebsites dstdomain -i "/blacklists/allowedwebsite/domains" > acl allowwebsites_url url_regex -i "/blacklists/allowedwebsite/url" > http_access allow allowwebsites > http_access allow allowwebsites_url > acl shopping dstdomain -i "/etc/squid/shopping.txt" > acl social_networking dstdomain -i "/blacklists/social/social.networking" > acl youtube dstdomain -i .youtube.com > http_access allow Safe_ports pro1 pro2 pro3 pro4 pro5 pro6 webvip Incorrect use of "Safe_ports" security check. Correct usage is to deny access to all *unsafe* ports. They are unsafe because HTTP can be smuggled within the ports native protocol to attack your proxy. Once the correct security protections for Safe_port and CONNECT tunnels have been moved up the top remove the "Safe_ports" check from this line. This line is also very odd in another way. ACL tests in a single line are AND'ed together - so this means the request must be from a user who is: authenticated AND a member of group pro1 AND pro2 AND pro3 AND pro4 AND pro5 AND pro6 AND webvip This hints at what your main helper problem is. The above line requires 7 group helper lookups *per request*. The winbind helper has a maximum of 200 simultaneous connections. This line alone will limit your proxy just under 30 new visitors per second (that becomes 60 lookups/sec before queue overload). The helper result caching will help a lot, but you also have a LOT of other group checks being made and 800 users. > http_access allow youtube pro5 > http_access allow youtube pro6 > http_access allow youtube webvip > http_access deny youtube > http_access allow shopping pro5 > http_access allow shopping pro6 > http_access allow shopping webvip > http_access deny shopping Optimization hint: "youtube" and "shopping" have the same allow/deny criteria. It would be worth combining them into one ACL. > http_access allow social_networking pro2 > http_access allow social_networking pro4 > http_access allow social_networking pro6 > http_access allow social_networking webvip > http_access deny social_networking > acl porn_site1 dstdomain "/etc/squid/blacklists/porn/domains.txt" > acl porn_site2 dstdom_regex -i "/etc/squid/blacklists/porn/expressions" > acl porn_site3 dstdom_regex -i "/etc/squid/blacklists/porn/urls.txt" > acl audio_video1 dstdomain "/etc/squid/blacklists/audio-video/urls.txt" > ###################### THERE ARE TOO MANY acls and http_access , so not > bothering with vast linux I will bet a lot of those ACLs are also calling the group helper too yes? > http_access allow liquorinfo webvip > http_access deny liquorinfo > http_access allow ad_auth > http_access allow auth Once you have removed ad_auth ACL, this becomes: http_access allow auth http_access allow auth I hope you can see how redundant that is. Also, its very likely that the "allow auth" is a useless operation after a great many group checks have also performed authentication. That "TOO MANY acls and https_access" list you omitted will be needed to determine that. > http_access allow sq1 sq2 > acl NTLMUsers proxy_auth REQUIRED You already have an ACL named "auth" which performs authentication. The above line is not being used in any way. Remove it. > http_access deny !Safe_ports > http_access deny CONNECT !SSL_ports These are basic security protection against Denial of Service and other types of protocol smuggling attacks. They only work when they are used *above* your custom "allow" rules. Move these two lines above your "http_access allow google" line. > http_port 8080 > hierarchy_stoplist cgi-bin ? The above line is not useful these days. Remove it. > cache_effective_user squid > cache_dir aufs /var/spool/squid 20384 32 512 > cache_mem 50 MB > cache_replacement_policy heap LFUDA > cache_swap_low 85 > cache_swap_high 95 > maximum_object_size 5 MB > maximum_object_size_in_memory 50 KB > ipcache_size 5240 > ipcache_low 90 > ipcache_high 95 > cache_mgr amit > cachemgr_passwd **** I hope that was not your real cachemgr password you just published on a public mailing list. > acl SSL_ports port 443 The above is a duplicate config line. Remove it. > http_access allow CONNECT SSL_ports > coredump_dir /var/spool/squid > refresh_pattern ^ftp: 1440 20% 10080 > refresh_pattern ^gopher: 1440 0% 1440 > refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 > refresh_pattern . 0 20% 4320 > url_rewrite_program /usr/local/bin/squidGuard -c > /usr/local/squidGuard/squidGuard.conf > Now, as to solving your problem: 1) Clean up your config. Reduce the amount of redundant or unused things. I've mentioned a few above. 2) Run "squid -k parse" and fix any other problems it highlights. 3) optimize your ACls and http_access rules. I've mentioned a few, such as moving the main security checks to the top so DoS traffic does not put load on the helpers and other ACLs. I believe though that you will probably find Squid works much better having the following access controls pattern: " http_access deny !Safe_ports http_access deny CONNECT !SSL_ports # if they are not authenticated, they will not be in a group http_access deny !auth # assuming that webvip are the group with full access? http_access allow webvip # your long list of per-site group check ACLs go here ... # this is where defining the LAN ranges correctly comes in. # note that users have authenticated simply to get near here http_access allow localnet http_access deny all " 4) consider an upgrade to Squid 3.4+. The "notes" ACL type offers much more efficient ACL testing with a custom group lookup helper. The all-of and any-of ACL types can also much reduce your http_access lines. HTH Amos _______________________________________________ squid-users mailing list squid-users@xxxxxxxxxxxxxxxxxxxxx http://lists.squid-cache.org/listinfo/squid-users
