Amos: Thanks... this helps a lot.
I just have one question regarding... this:
client_persistent_connections off
server_persistent_connections off
If I enable this, my webservers starts having high load. The httpds in
them never closes any connection, and soon I run out of childs in the
webservers.. and my terminal servers too. I have aprox 70 users per
terminal server, if I set the persistent connections, everything goes
to hell.
Besides that, I run out of file descriptors.
Do you think that if I enable this back, the traffic generated by the
ntlm will be reduced? Do I need both options on?
Maybe I can activathe them back, increase the filedescriptors a lot,
and see if I cant force the persistent connection on the webservers to
close instead of keep alive...
What do you think?
Tnxs in advance.
Amos Jeffries <squid3@xxxxxxxxxxxxx> escribió:
Guido Marino Lorenzutti wrote:
Hi people: I have a squid using ntlm to authenticate the users. I
also use a external acl but im running out of ideas to make it run
faster.
If I disable the ntlm everything works very well, and the cache
hits increses a lot.
I found that the squids asks a lot for username and passwords to
the winbind, and the winbind asks everytime to my pdc. This
generates a lot of traffic between them and a high load on the pdc.
Every hit on any page, squid ask to the winbind for the username
and password. Is this the expected behavior? Is there any way to
reduce (caching maybe?) this? I didn't find a solution in the
winbind, to stop asking the credentials to the pdc.
I have a terminal server enviroment, so where you see 69 clients
the are in fact more than 500 users.
This is my output of squidclient mgr:info
Squid Object Cache: Version 2.6.STABLE5
Start Time: Fri, 05 Feb 2010 07:21:21 GMT
Current Time: Sat, 20 Feb 2010 03:01:08 GMT
Connection information for squid:
Number of clients accessing cache: 69
Number of HTTP requests received: 11790881
Number of ICP messages received: 0
Number of ICP messages sent: 0
Number of queued ICP replies: 0
Number of HTCP messages received: 0
Number of HTCP messages sent: 0
Request failure ratio: 0.00
Average HTTP requests per minute since start: 552.5
Average ICP messages per minute since start: 0.0
Select loop called: 154266917 times, 8.300 ms avg
Cache information for squid:
Request Hit Ratios: 5min: 50.5%, 60min: 18.3%
Byte Hit Ratios: 5min: 14.1%, 60min: 26.3%
Request Memory Hit Ratios: 5min: 0.0%, 60min: 10.0%
Request Disk Hit Ratios: 5min: 19.7%, 60min: 21.9%
Storage Swap size: 7833612 KB
Storage Mem size: 409452 KB
Mean Object Size: 19.26 KB
Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
HTTP Requests (All): 0.00919 0.03066
Cache Misses: 0.35832 0.44492
Cache Hits: 0.01164 0.01847
Near Hits: 0.33943 0.37825
Not-Modified Replies: 0.00286 0.00405
DNS Lookups: 0.09117 0.10906
ICP Queries: 0.00000 0.00000
Resource usage for squid:
UP Time: 1280387.834 seconds
CPU Time: 5238.387 seconds
CPU Usage: 0.41%
CPU Usage, 5 minute avg: 0.07%
CPU Usage, 60 minute avg: 0.05%
Process Data Segment Size via sbrk(): 561092 KB
Maximum Resident Size: 0 KB
Page faults with physical i/o: 4
Memory usage for squid via mallinfo():
Total space in arena: 561092 KB
Ordinary blocks: 555876 KB 13964 blks
Small blocks: 0 KB 0 blks
Holding blocks: 1744 KB 4 blks
Free Small blocks: 0 KB
Free Ordinary blocks: 5215 KB
Total in use: 557620 KB 99%
Total free: 5215 KB 1%
Total size: 562836 KB
Memory accounted for:
Total accounted: 511637 KB
memPoolAlloc calls: 1443436295
memPoolFree calls: 1441310223
File descriptor usage for squid:
Maximum number of file descriptors: 1024
Largest file desc currently in use: 268
Number of file desc currently in use: 261
Files queued for open: 0
Available number of file descriptors: 763
Reserved number of file descriptors: 100
Store Disk files open: 2
IO loop method: epoll
Internal Data Structures:
407686 StoreEntries
34175 StoreEntries with MemObjects
34170 Hot Object Cache Items
406635 on-disk objects
This is the output of squidclient mgr:ntlmauthenticator
(warning: the avg service time is with NO users, when everyone is
using it the avg service time peeks the 1000 msec. YES 1K msec).
NTLM Authenticator Statistics:
program: /usr/bin/ntlm_auth
number running: 200 of 200
requests sent: 2500498
replies received: 2500498
queue length: 0
avg service time: 19.24 msec
# FD PID # Requests Flags Time Offset Request
1 12 17113 168619 0.046 0 (none)
2 13 17114 62644 0.055 0 (none)
3 14 17118 31007 0.076 0 (none)
4 15 17120 15188 0.094 0 (none)
5 16 17121 5759 0.093 0 (none)
6 17 17122 2845 0.071 0 (none)
7 18 17124 1572 0.524 0 (none)
8 19 17125 891 0.533 0 (none)
9 21 17130 486 0.584 0 (none)
10 22 17131 302 0.647 0 (none)
11 23 17132 194 0.741 0 (none)
12 24 17135 127 0.818 0 (none)
13 25 17137 84 0.756 0 (none)
14 26 17138 56 0.898 0 (none)
15 27 17143 46 0.954 0 (none)
16 28 17149 36 1.002 0 (none)
17 29 17155 24 1.125 0 (none)
18 30 17161 22 1.094 0 (none)
19 31 17162 16 1.252 0 (none)
20 32 17165 10 5.137 0 (none)
21 33 17167 8 4.807 0 (none)
22 34 17168 4 1.470 0 (none)
23 35 17169 4 1.522 0 (none)
24 36 17170 2 1.185 0 (none)
25 37 17171 2 0.613 0 (none)
26 38 17172 2 0.839 0 (none)
27 39 17173 0 0.000 0 (none)
Any ideas in how to improve this scenario?
This is the squid.conf
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
http_port 3128
#debug_options ALL,1 33,2
log_fqdn off
cache_store_log none
useragent_log none
cache_log /var/log/squid/cache_log.log
access_log /var/log/squid/access.log
error_directory /usr/share/squid/errors/Spanish
emulate_httpd_log on
offline_mode off
strip_query_terms on
httpd_suppress_version_string on
auth_param ntlm program /usr/bin/ntlm_auth
--helper-protocol=squid-2.5-ntlmssp --domain=MYDOMAIN
auth_param ntlm children 200
auth_param ntlm keep_alive on
authenticate_ttl 60 seconds
authenticate_ip_ttl 2 minutes
authenticate_cache_garbage_interval 10 seconds
Seems a bit extreme to be running the garbage collection 10 seconds.
It happens as needed on top of this.
The defaults are measured in hours and user browsing times are
usually longer than minutes.
auth_param basic program /usr/bin/ntlm_auth
--helper-protocol=squid-2.5-basic --domain=MYDOMAIN
auth_param basic children 10
auth_param basic realm Squid proxy-caching web server
auth_param basic credentialsttl 2 hours
external_acl_type ldap_group ttl=1200 children=25 %LOGIN
/usr/lib/squid/squid_ldap_group -b "GROUPDN" -f "MYFILTER" -h
LDAPSERVER -v3 -S -P
negative_ttl 5 minutes
This is not really a good idea.
It will extend the period of outage for every service failure and
may hose the network access to a website for 5 minutes following a
single client page error.
positive_dns_ttl 5 hours
negative_dns_ttl 1 minutes
Please don't play with DNS TTLs unless you know 100% how they will
affect things.
half_closed_clients off
connect_timeout 3 seconds
cache_dir aufs /var/spool/squid 9000 16 256
cache_swap_low 85
cache_swap_high 95
maximum_object_size 81920 KB
maximum_object_size_in_memory 300 KB
cache_mem 400 MB
fqdncache_size 6144
cache_replacement_policy lfuda
memory_replacement_policy lru
pipeline_prefetch off
client_persistent_connections off
server_persistent_connections off
Persistent connections are REQUIRED for NTLM and related
connection-based auth to be used efficiently.
NTLM auth against the proxy requires persistent client connections,
pass-thru to web servers requires both and the connection pinning
feature as well.
visible_hostname myproxy.mydomain
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
acl all src all
acl lan_10_8 src 10.8.0.0/255.255.0.0
acl lan_10_8 src 10.8.0.0/16
acl webservers dst 10.8.50.220/255.255.255.255
10.8.50.221/255.255.255.255 10.8.50.222/255.255.255.255
10.8.50.223/255.255.255.255
acl webservers dst 10.8.50.220 10.8.50.221 10.8.50.222 10.8.50.223
acl nomsnurl dstdomain "/etc/squid/nomsn"
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl localhost src 127.0.0.1
acl SSL_ports port 443 563 1863 6667 4430
acl Safe_ports port 80 # http
acl Safe_ports port 443 563 # https, snews
acl auth proxy_auth REQUIRED
acl noinet external ldap_group noinet
acl linuxadmin external ldap_group linuxadmin
acl nomsn external ldap_group nomsn
acl dummy src 0.0.0.0/0.0.0.0
acl dummy src all
Amos
--
Please be using
Current Stable Squid 2.7.STABLE8 or 3.0.STABLE24
Current Beta Squid 3.1.0.16
