I'm still confused about squid's behavior in 3.2.7 concerning credentials-caching and the order of the http_access-directive. Does someone has an explanation for squids behavior (look the questions in this post..) and does squid even cache the negotiate-credentials?? Many thanks. Tom On Thu, Feb 28, 2013 at 1:45 PM, Tom Tom <tomtux007@xxxxxxxxx> wrote: > On Tue, Feb 26, 2013 at 2:31 AM, Amos Jeffries <squid3@xxxxxxxxxxxxx> wrote: >> On 25/02/2013 8:27 p.m., Tom Tom wrote: >>> >>> I've attached both cache-traces (squid 3.2.7). >>> >>> "without_407.txt" has the following configuration: >>> ... >>> ... >>> external_acl_type SQUID_KERB_LDAP ttl=7200 children-max=10 >>> children-startup=1 children-idle=1 negative_ttl=7200 %LOGIN >>> /usr/local/squid/libexec/ext_kerberos_ldap_group_acl -g >>> "INTERNET_USERS" >>> acl INTERNET_ACCESS external SQUID_KERB_LDAP >>> acl AUTHENTICATED proxy_auth REQUIRED >>> http_access deny !INTERNET_ACCESS >>> http_access deny !AUTHENTICATED >>> http_access allow INTERNET_ACCESS AUTHENTICATED >>> http_access allow localhost >>> http_access deny all >>> ... >>> ... >> >> >> Note for anyone else reading this: >> The above was a copy-n-paste typo. The without-407 config has no >> AUTHENTICATED access control definition. > I think that the "without-407.txt" file HAS AUTHENTICATED access > control definition. The "without-407.txt" means (for me), that there > is NO 407 response. So this trace should show the behavior, where > squid accepts the request without firing the 407. > 2013/02/25 07:26:30.936 kid1| Acl.cc(250) cacheMatchAcl: > ACL::cacheMatchAcl: cache hit on acl 'AUTHENTICATED' (0xaafa60) > 2013/02/25 07:26:30.936 kid1| Acl.cc(312) checklistMatches: > ACL::ChecklistMatches: result for 'AUTHENTICATED' is 1 > > Is it possible, that squid uses his already filled up > credentials-cache (mapping of ie. username+ip with > kerberos-credentials) in the "without-407.txt"-trace? (look above: > cache hit on acl 'AUTHENTICATED'...) > > > >> >> >>> In this case, the access.log shows the following: >>> Mon Feb 25 08:14:23 2013 15 10.X.X.X TCP_REFRESH_UNMODIFIED/304 >>> 283 GET http://imagesrv.adition.com/banners/750/683036/dummy.gif >>> user@xxxxxxxxxxx HIER_DIRECT/217.79.188.10 image/gif >>> >>> >>> >>> "with_407.txt" has the following configuration: >>> ... >>> ... >>> external_acl_type SQUID_KERB_LDAP ttl=7200 children-max=10 >>> children-startup=1 children-idle=1 negative_ttl=7200 %LOGIN >>> /usr/local/squid/libexec/ext_kerberos_ldap_group_acl -g >>> "INTERNET_USERS" >>> acl INTERNET_ACCESS external SQUID_KERB_LDAP >>> acl AUTHENTICATED proxy_auth REQUIRED >>> http_access deny !INTERNET_ACCESS >>> http_access deny !AUTHENTICATED >>> http_access allow INTERNET_ACCESS >>> http_access allow localhost >>> http_access deny all >>> ... >>> ... >>> >>> >>> In this case, the access.log shows the following: >>> Mon Feb 25 08:14:22 2013 0 10.X.X.X TCP_DENIED/407 4136 GET >>> http://imagesrv.adition.com/banners/750/683036/dummy.gif - HIER_NONE/- >>> text/html >>> Mon Feb 25 08:14:22 2013 56 10.X.X.X TCP_REFRESH_UNMODIFIED/304 >>> 354 GET http://imagesrv.adition.com/banners/750/683036/dummy.gif >>> user@xxxxxxxxxxx HIER_DIRECT/217.79.188.10 image/gif >>> >>> The only different between config1 and config2 is the >>> "AUTHENTICATED"-flag on the "http_access allow INTERNET_ACCESS" line. >>> >>> Many thanks. >>> Kind regards, >>> Tom >> >> >> Thank you. I have an explanation for you. But I'm not exactly happy with it >> how it is working in practice ... >> >> >> The difference is the fine boundary between authentication and authorization >> with a couple of factors (* marked) leading into the behaviour: >> >> * Negotiate auth protocol credentials, once delivered, are tied to the TCP >> connection state. >> >> Seems simple at face value, but HTTP is stateless - ie there is no TCP >> connection state persistence between requests. So we have to jack up all >> sorts of pinning an connection persistence just to make our HTTP connections >> stateful for each client using Negotiate. >> >> Keep in mind that Negotiate satetfulness is the anomally haunting us here. >> >> >> * both your traces look like the connection is already setup and had >> previous traffic so the request credentials should be checked against the >> existing validated credentials. > Yes, before I took the traces, I made a few requests before (to > enforce the behavior). >> >> >> * external_acl_type with %LOGIN is simply an authorization test. Are the >> credentials we have allowing permission for this request to continue? >> yes/no. >> >> IMPORTANT: authorization takes no notice of validity of the credentials. Or >> of their accuracy. Only existence and permissions assigned to it matters. >> >> It locates credentials anywhere it can and uses those. In *both* your traces >> the external ACL test locates credentials already tied to the TCP connection >> state, which it validates with the helper as _authorized_. The helper says >> OK, as you would expect since these connection credentials were okay on >> previous traffic. >> >> This is fine and consistent with external ACL definition as an authorization >> API. It is not intended for use as authenticator. >> >> >> * proxy_auth ACL is an authentication test. Is the request user who they >> claim to be? yes/no. >> >> For this the request requires credentials which can be verified against >> either the backend or against the tied credentials. To say yes the >> credentials for this request are valid. >> Your without-407 trace is the one doing proxy_auth test and detecting that >> the HTTP request in fact has *no* credentials --> therefore fail with 407 to >> get some. > No, this is in the "with-407.txt"-trace. In the > "without-407.txt"-trace squid doesn't fail with 407. > > >> >> >> In summary, >> * using only an external_acl_type login check with Negotiate (or NTLM) >> authentication is equivalent to performing a IP-based authentication bypass, >> but limited to bypass only on the TCP connection which has already been >> authenticated. >> >> All nice and consistent with the individual ACL tests and what they mean. >> The part I'm not happy with is that with this config Squid lets requests >> through without validating every single one has a credentials token matching >> the tied connection credentials. I will have to look it up in details and >> double-check but AFAIK that test SHOULD be done regardless of what ACLs you >> configure just to ensure no requests are injected into the stream by an >> attacker. For now I do recommend using explicit proxy_auth tests before >> anything which requries credentials. > > In the trace/behavior, which provides me the 407 for each request, > there are "freeing"-tasks: > 2013/02/25 07:36:51.392 kid1| UserRequest.cc(311) HandleReply: > authenticated user user@xxxxxxxxxxx > 2013/02/25 07:36:51.392 kid1| User.cc(101) absorb: auth_user > '0xe6bef0*3' into auth_user '0x15a58b0'. > 2013/02/25 07:36:51.392 kid1| User.cc(14) ~User: doing nothing to > clear Negotiate scheme data for '0xe6bef0' > 2013/02/25 07:36:51.392 kid1| User.cc(154) ~User: Freeing auth_user '0xe6bef0'. > > The things, which are unclear for me are: > > <<config, which doesn't throw a 407 (which is working without 407)>> > ... > http_access deny !AUTHENTICATED > http_access allow INTERNET_ACCESS AUTHENTICATED > ... > - Does this config "activates" a kind of persistent credentials-cache? > If yes, what are the persistent-parameters (ie. ip-address, port, > user..)? With "squidclient mgr:username_cache", the username entry > persists until the the TTL expires or squid is reloading. If no > credentials-cache is present, why does squid accepts the request > without firing a 407 (although there is no proxy-authorization > provided in the request-header)? > - TCP-Traces are not showing any 407 (only in the the first > initial-request!) or any "proxy-authorization:"-headers in the > http-packet (for my example-request: > http://imagesrv.adition.com/banners/750/683036/dummy.gif) > > > > <<config, which ends in a 407>> > http_access allow INTERNET_ACCESS > ... > - This makes sens for me to fire a 407, if the client doesn't provide > the necessary credentials. EXCEPT: Squid should use a > credential-cache. > - With this config, after EVERY request, "squidclient > mgr:username_cache" resets the TTL counter. Does this probably relate > with the "freeing"-tasks above? > > > When I change the line from 'http_access allow INTERNET_ACCESS > AUTHENTICATED' to 'http_access allow AUTHENTICATED INTERNET_ACCESS', > then I always got a 407. Squidclient then also resets the TTL > respectively cleans the entry and creates a new one...). > > Do I understand correctly, that in your point of view squid should > always throw a 407, If no credentials are provided within the > requests? Even squid has an entry in his credential/username cache? > Does the "username_cache" also means "credentials-cache"? Is there > generally a credentials-cache existing? > > Many thanks for your help and explanations. > Tom > > >> >> >> PS. I note that your ealier messages about the without-AUTHENTICATED traffic >> being the ones getting 407 was incorrect. Which brings me to the difference >> between 3.1 and 3.2. >> If the squid-3.1 proxy_auth test was using the connection tied credentials >> and permitting requests through without validating the request header >> properly, you would see the without-407 behaviour. That would be a bug in >> 3.1 which was fixed in 3.2. >> >> Amos
