Search squid archive

Re: Authentication Logging/Debugging

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 17.04.2012 09:03, Chris Waters wrote:
I am working some authentication/performance issues with Squid 3.1.19 on
Ubuntu with NTLM via winbind and a filtering upstream service (proxy
chaining). Is there a method for turning on more verbose logging that
either shows the object the authentication parts are processing on or
interleaving the access log and debug log so that its apparent which
authentication request belongs to which object?  What I am trying to
determine is if there are authentication issues or if it's the upstream
causing the latency.

You can use the auth_user_request/AuthUserRequest hex IDs to determine creation (start) and result (finish) lookup timings. Those objects store the handshake and validation state info and are thus the best indicator available of what is going on.


I am not specifically talking about turning specific logging sections on (like 28,9 29,9 for example), but a better method of matching what's being
requested and what's being done to authenticate the request.


NTLM does not authenticate *requests*. Please be clear on that.

It hijacks and violates the HTTP layer headers to authenticate the TCP connection layer packets. Then assumes (incorrectly) that all requests sent over the TCP connection are from the same end-user. In order to cope with that bad assumption Squid is forced to disable all HTTP multiplexing abilities on connections sighted using NTLM or Negotiate auth (we call this "pining").

* This pinning is #3 major cause of latency issues with NTLM in proxy chaining. Since it locks two socket connections per end-user at each hop along the HTTP chain (creating a limit of ~64K users per proxy listening IP).

As a side effect; other than the first-request rejections there is nothing to correlate access.log records with NTLM authentication. You need to create a custom log format with both %>a and %>p to get the client IP:port details about the TCP connection and see what requests are taking place on it. Then map those requests to a particular state order for the credentials contained within, then to the AuthUserRequest object managing those credentials. Fun times.


I am also open to suggestion as to better methods to diagnose these kinds
of issues as well, so please, if you have an idea, let's here it.

Also look at the size of body/entity/data in requests taking part in the NTLM handshake. NTLM requires the HTTP request to be repeated for each of the handshake steps. This is not too bad on a high speed link, or with small fetch request. But when uploading say a multi-MB (or GB) object it can take down a network.

* bandwidth + CPU + time wasted discarding POST/PUT bodies is #1 major cause of latency from NTLM.


Also, look at winbind concurrent lookup load when latency is observed. Winbind has an absolute limit somewhere around 256 concurrent authentications. Each authentication takes the *full* handshake worth of HTTP request round trips to complete. Additional lookups (TCP connections) are blocked until winbind can service them.

 * winbind queue limits is #2 major cause of latency with NTLM.



Two additional things to be aware of:
* NTLM is officially been obsoleted by MS for some years now, to be completely End-of-Lifed with WindowsXP and none of their new software supports it by default. * Negotiate/Kerberos is supported by all the new software and was designed to avoid the above latency and bandwidth waste issues. When comparing auth speeds between MS software from before and after 2007, default use of Kerberos is often the reason newer ones work far faster and more reliably.


Amos



[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux