Re: Squid not communicating with Helper Processes

Amos Jeffries <squid3@xxxxxxxxxxxxx> · Tue, 29 Nov 2011 00:30:01 +1300

On 28/11/2011 11:29 p.m., Christian Zink wrote:
Finally i've got it!
After searching in the buglist i found a case describing Problems if you use both authentication and external helpers. 
So i tried to disable authentication and use only squid_ldap_group external helper:

auth_param      digest program /usr/lib64/squid/digest_ldap_auth -H ldap://127.0.0.1 -v 3 -e -b "ou=0500,dc=drv,dc=drv" -u "uid" -A "userPassword" -D "uid=digestreader,dc=drv,dc=drv" -W "/etc/squid/digestreader_cred"
auth_param      digest children 1	
auth_param      digest realm Proxy
#acl ldap proxy_auth REQUIRED
external_acl_type ldap_group %LOGIN /usr/lib64/squid/squid_ldap_group -H ldap://127.0.0.1 -v 3 -b ou=groups,dc=drv,dc=drv -f (&(memberuid=%u)(cn=%g)) -D uid=digestreader,dc=drv,dc=drv -w digestpw -dacl ldap_download
external ldap_group downloadacl ldap_browse
external ldap_group browse
cache_peer 10.10.1.254 parent 9986 0 no-query no-digest name=download.proxy
cache_peer_access download.proxy deny !ldap_download
cache_peer 10.10.1.254 parent 9985 0 no-query no-digest name=browse.proxy
cache_peer_access browse.proxy allow ldap_browse
http_access allow ldap_download
http_access allow ldap_browse

And voila, not only ldap group check worked, but it also still needed authentication first! I don't know if its a bug, or intended ... but if you use "acl ldap proxy_auth REQUIRED"  there will be no other requestto an external helper again ... without it it works as desired. In my eyes maybe its a feature to also authenticate if i use external_helper,but it should also work with this acl. Perhaps someone could explain the ideas behind this.

It is exactly what you configured within th eSquid Access Control 
designs.  %LOGIN means " authenticated username". Implying that one must 
be logged in for the ACL to be tested. When possible Squid will lookup 
requirements for ACL tests, such as this username.

The difference between the above working config and the earlier failing 
config. Is that in the "failing" one you are allowing the user 
http_access depending only on them having valid credentials (of any user 
in any group). Then deciding the peer based on group check. The 
cache_peer_access is a "fast" category check which cannot wait for 
external ACL to do the group lookup. So the group is not known by 
cache_peer_access, and cannot wiat to find it --> prevented all peers 
being used.

Since you dropped "http_access allow ldap" Squid can now reach the 
"http_access allow ldap_download" line which performs group lookup and 
stores the results for the cache_peer_access to use.

Since you are controlling things based on groups rather than username 
the only use your config has for ACL "ldap", is in a deny ACL rejecting 
bad logins. For example:
   # reject failed logins
   http_access deny !ldap
   # permit either one of several allowed groups
   http_access allow ldap_download
   http_access allow ldap_browse

Amos

Hi,
i have a strange problem driving me mad. I set up a fresh RHEL 6.1 System and installedLDAP and Squid. I want do authenticate users and contol the internet access depending on groups. 
Ldap auth with digest_ldap_auth works fine, but i can't get the squid_ldap_group helper to work.
My conf:
#debug_options ALL,9debug_options ALL,1 28,3 82,9 84,9
hosts_file /etc/hosts
acl manager proto cache_objectacl localhost src 127.0.0.1/32
acl SSL_ports port 443 563 888 7002 8443 10000
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker>  acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT>  http_access allow manager localhost
http_access deny manager>  http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
auth_param      digest program /usr/lib64/squid/digest_ldap_auth -H ldap://127.0.0.1 -v 3 -e -b "ou=0500,dc=drv,dc=drv" -u "uid" -A "userPassword" -D "uid=digestreader,dc=drv,dc=drv" -W "/etc/squid/digestreader_cred"
auth_param      digest children 1
auth_param      digest realm Proxy
acl ldap proxy_auth REQUIRED

#external_acl_type ldap_group %LOGIN /usr/lib64/squid/squid_ldap_group -H ldap://127.0.0.1 -v 3 -b ou=groups,dc=drv,dc=drv -f (&(memberuid=%u)(cn=%g)) -D uid=digestreader,dc=drv,dc=drv -W "/etc/squid/digestreader_cred"
external_acl_type ldap_group children=1 %LOGIN /usr/lib64/squid/test.sh
acl ldap_download external ldap_group download
acl ldap_browse external ldap_group browse

cache_peer 10.10.1.254 parent 9986 0 no-query no-digest name=download.proxy
cache_peer_access download.proxy allow ldap_download
cache_peer 10.10.1.254 parent 9985 0 no-query no-digest name=browse.proxy
cache_peer_access browse.proxy allow browse

http_access allow ldap>  http_access deny all
http_port 8086

The Problem:
Squid doesnt talk to the Helper Processes! That's all i can see in logs:
2011/11/23 17:07:34.219| ACLChecklist::preCheck: 0x7fff8c40cc70 checking 'cache_peer_access download.proxy allow ldap_download'
2011/11/23 17:07:34.219| ACLList::matches: checking ldap_download
2011/11/23 17:07:34.219| ACL::checklistMatches: checking 'ldap_download'
2011/11/23 17:07:34.219| aclMatchExternal: acl="ldap_group"
2011/11/23 17:07:34.219| aclMatchExternal: ldap_group("v990493 download") = lookup needed
2011/11/23 17:07:34.219| aclMatchExternal: "v990493 download": entry=@0, age=0
2011/11/23 17:07:34.219| aclMatchExternal: "v990493 download": queueing a call.
2011/11/23 17:07:34.219| aclMatchExternal: "v990493 download": return -1.
2011/11/23 17:07:34.219| ACL::ChecklistMatches: result for 'ldap_download' is -1
2011/11/23 17:07:34.219| aclmatchAclList: 0x7fff8c40cc70 returning false (AND list entry failed to match)
2011/11/23 17:07:34.219| aclmatchAclList: async=0 nodeMatched=0 async_in_progress=0 lastACLResult() = 0 finished() = 0

While this is repeated endlessly i straced the helper Process ... nothing! I also wrote a dummy Helper, also nothing.Tcpdump on localhost i see the packets from digest_ldap_auth to ldap. Squids talking to digest_ldap_auth over Unix Pipe, that works, and form digest_ldap_auth to ldap over 127.0.0.1 works to,but not from Squid to the Helper although there is an TCP Connection:
squid      1858     squid    8u     IPv6              47834      0t0        UDP *:54597
squid      1858     squid   14u     IPv6              47840      0t0        TCP [::1]:38965->[::1]:45367 (ESTABLISHED)
squid      1858     squid   15u     IPv6              47842      0t0        TCP *:d-s-n (LISTEN)
test.sh   10617     squid    0u     IPv6              47841      0t0        TCP [::1]:45367->[::1]:38965 (ESTABLISHED)
test.sh   10617     squid    1u     IPv6              47841      0t0        TCP [::1]:45367->[::1]:38965 (ESTABLISHED)

What i tried so far:
- the squid_ldap_group works on the shell, piping Username&  Group result in OK/ERR depending on the ldap group membership
- no activity in strace on squid_ldap_group, but on digest_ldap_auth
- no Packets seen with tcpdump on localhost, except from digest_ldap_auth
- tried many different options of external_acl_type ...
- no iptables active&  SELinux Permissive
Probably it's a really simple solution, like an internal acl not allowing network access to localhost, but i can't see it and its driving me nuts !!!!

Squid Cache: Version 3.1.10configure options:  '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--target=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--exec_prefix=/usr' '--libexecdir=/usr/lib64/squid' '--localstatedir=/var' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--with-logdir=$(localstatedir)/log/squid' '--with-pidfile=$(localstatedir)/run/squid.pid' '--disable-dependency-tracking' '--enable-arp-acl' '--enable-follow-x-forwarded-for' '--enable-auth=basic,digest,ntlm,negotiate' '--enable-basic-auth-helpers=LDAP,MSNT,NCSA,PAM,SMB,YP,getpwnam,multi-domain-NTLM,SASL,DB,POP3,squid_radius_auth' '--enable-ntlm-auth-helpers=smb_lm,no_check,fakeauth' '--enable-digest-auth-helpers=password,ldap,eDirectory' '--enable-negotiate-auth-helpers=squid_kerb_auth' '--enable-external-acl-helpers=ip_user,ldap_group,session,unix_group,wbinfo_group' '--enable-cache-digests' '--enable-cachemgr-hostname=localhost' '--enable-delay-pools' '--enable-epoll' '--enable-icap-client' '--enable-ident-lookups' '--enable-linux-netfilter' '--enable-referer-log' '--enable-removal-policies=heap,lru' '--enable-snmp' '--enable-ssl' '--enable-storeio=aufs,diskd,ufs' '--enable-useragent-log' '--enable-wccpv2' '--enable-esi' '--with-aio' '--with-default-user=squid' '--with-filedescriptors=16384' '--with-dl' '--with-openssl' '--with-pthreads' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'target_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fpie' 'LDFLAGS=-pie' 'CXXFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fpie' --with-squid=/builddir/build/BUILD/squid-3.1.10
Linux squidproxy51.drv.drv 2.6.32-131.17.1.el6.x86_64 #1 SMP Thu Sep 29 10:24:25 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux