Re: -k reconfigure to slow

[Guido Marino Lorenzutti <glorenzutti@xxxxxxxxxxxxxxxx>]

Imagine that!
"... Well people, i have to change a mail address in the whitelist of  
the mailserver, everyone STOP SENDING MAILS that the server needs to  
be restarted.. and ALL the childs... so.. HOLD ON THE SEND BUTTON... "  
(WFT?!)

I have to apply all the changes at midnight... With one, or two  
minutes of downtime in the squid I lost a lot of connections to my  
webaps and external pages...

I can measure the problem with iptables, but Im sure that at least I  
lost 300 requests in that time.

And if you use a reverse proxy.. well.. prepare to cry..

It would be great to have a tool to tests urls with a set of acls, so  
you can test before apply.

Jorge Armando Medina <jmedina@xxxxxxxxxxxxxxx> escribió:

> Guido Marino Lorenzutti wrote:
> > I don't understand why it works like this! But it seems that this is
> > like the squid works... maybe we should also ask why the childs needs
> > to be restarted, if you change an acl... this sucks.
> I agree, I have same problem, whenever I add a ACL I have to
> reconfigure, and for a few seconds users can't access the proxy, then
> they have to reload the page :(.
> > I don't remember any other app with this popularity that works like
> > this...
> >
> > instead of working better with 64bits, it works worst? I have many
> > squids, I don't recall having one in 32bits to compare...
> >
> > Malte Schröder <maltesch@xxxxxx> escribió:
> >
> > > Hello,
> > > I also have one thing to add: I think it is worse on 64bit than on 32bit
> > > linux. My guess is that closing and forking all those processes is just
> > > too expansive.
> > >
> > >
> > > On Wed, 04 Nov 2009 16:39:08 -0300
> > > Guido Marino Lorenzutti <glorenzutti@xxxxxxxxxxxxxxxx> wrote:
> > >
> > > > Sorry, can't help you with the problem. But I have a question
> > > > instead :)
> > > >
> > > > Where did you get the ip-user-helper.pl ?
> > > >
> > > > Tryin' to help... i have also 1k connections, squid 2.7 and it takes
> > > > that time too... i found that if i reduce the number of helpers it
> > > > starts up faster.
> > > >
> > > > I have 180 NTLM, 10 basic, and 75 squid_ldap_group childrens..
> > > > With less, it starts faster.
> > > >
> > > > If you do squid -k reconfigure and watch the proceess list, you will
> > > > see that the squid dosen't work until all the childrens are running...
> > > > it would be great that the squids starts as soon as ONE child per type
> > > > is running... or to have a minimum childrens, and a maximum
> > > > childrens...
> > > >
> > > > Sorry for my english, never study :)
> > > >
> > > > Luis Daniel Lucio Quiroz <luis.daniel.lucio@xxxxxxxxx> escribió:
> > > >
> > > > > Le mardi 3 novembre 2009 22:50:58, Amos Jeffries a écrit :
> > > > >> Luis Daniel Lucio Quiroz wrote:
> > > > >> > HI squids,
> > > > >> >
> > > > >> > We have 2 squid server, one with load other with minimal (1-2
> > > > users).
> > > > >> > After doing a -k reconfigure, the loaded server delays 40
> > > > seconds, but
> > > > >> > unloaded 2 seconds. Look:
> > > > >> >
> > > > >> > Unloaded:
> > > > >> > 2009/11/03 19:01:14| Processing Configuration File:
> > > > /etc/squid/squid.conf
> > > > >> > (depth 0)
> > > > >> > 2009/11/03 19:01:14| Processing Configuration File:
> > > > /etc/squid/squid.acl
> > > > >> > (depth 1)
> > > > >> > 2009/11/03 19:01:14| WARNING: HTTP requires the use of Via
> > > > >> > 2009/11/03 19:01:14| Initializing https proxy context
> > > > >> > 2009/11/03 19:01:14| Store logging disabled
> > > > >> > 2009/11/03 19:01:14| User-Agent logging is disabled.
> > > > >> > 2009/11/03 19:01:14| Referer logging is disabled.
> > > > >> > 2009/11/03 19:01:14| DNS Socket created at 0.0.0.0, port 49328,
> > > > FD 8
> > > > >> > 2009/11/03 19:01:14| Adding nameserver 127.0.0.1 from
> > > > /etc/resolv.conf
> > > > >> > 2009/11/03 19:01:14| Adding domain sat.gob.mx from
> > > > /etc/resolv.conf
> > > > >> > 2009/11/03 19:01:14| Adding domain insys-corp.com.mx from
> > > > >> > /etc/resolv.conf 2009/11/03 19:01:14| helperOpenServers:
> > > > Starting 32/32
> > > > >> > 'squidGuard' processes 2009/11/03 19:01:15| helperOpenServers:
> > > > Starting
> > > > >> > 16/16 'digest_ldap_auth' processes
> > > > >> > 2009/11/03 19:01:15| helperOpenServers: Starting 32/32
> > > > 'squid_ldap_group'
> > > > >> > processes
> > > > >> > 2009/11/03 19:01:15| helperOpenServers: Starting 32/32
> > > > >> > 'ip-user-helper.pl' processes
> > > > >>
> > > > >> Wow! thats a LOT of helper processes for 2 users.
> > > > >>
> > > > >> They start fast enough though.
> > > > >>
> > > > >> > 2009/11/03 19:01:16| Accepting  HTTP connections at 0.0.0.0,
> > > > port 3128,
> > > > >> > FD 57. 2009/11/03 19:01:16| Accepting ICP messages at 0.0.0.0,
> > > > port 3130,
> > > > >> > FD 122. 2009/11/03 19:01:16| Accepting HTCP messages on port
> > > > 4827, FD
> > > > >> > 123. 2009/11/03 19:01:16| Accepting SNMP messages on port 3401,
> > > > FD 124.
> > > > >> >
> > > > >> > Loaded:
> > > > >> > 2009/11/03 18:59:05| Processing Configuration File:
> > > > /etc/squid/squid.conf
> > > > >> > (depth 0)
> > > > >> > 2009/11/03 18:59:05| Processing Configuration File:
> > > > /etc/squid/squid.acl
> > > > >> > (depth 1)
> > > > >> > 2009/11/03 18:59:05| WARNING: HTTP requires the use of Via
> > > > >> > 2009/11/03 18:59:05| Initializing https proxy context
> > > > >> > 2009/11/03 18:59:05| Store logging disabled
> > > > >> > 2009/11/03 18:59:05| User-Agent logging is disabled.
> > > > >> > 2009/11/03 18:59:05| Referer logging is disabled.
> > > > >> > 2009/11/03 18:59:05| DNS Socket created at 0.0.0.0, port 35507,
> > > > FD 8
> > > > >> > 2009/11/03 18:59:05| Warning: Could not find any nameservers.
> > > > Trying to
> > > > >> > use localhost
> > > > >> > 2009/11/03 18:59:05| Please check your /etc/resolv.conf file
> > > > >> > 2009/11/03 18:59:05| or use the 'dns_nameservers' option in
> > > > squid.conf.
> > > > >>
> > > > >> Oops! major problem with DNS on this server.
> > > > >>
> > > > >> > 2009/11/03 18:59:05| helperOpenServers: Starting 32/32
> > > > 'squidGuard'
> > > > >> > processes 2009/11/03 18:59:17| helperOpenServers: Starting 16/16
> > > > >> > 'digest_ldap_auth' processes
> > > > >> > 2009/11/03 18:59:23| helperOpenServers: Starting 32/32
> > > > 'squid_ldap_group'
> > > > >> > processes
> > > > >> > 2009/11/03 18:59:34| helperOpenServers: Starting 32/32
> > > > >> > 'ip-user-helper.pl' processes
> > > > >> > 2009/11/03 18:59:47| Accepting  HTTP connections at 0.0.0.0,
> > > > port 3128,
> > > > >> > FD 57. 2009/11/03 18:59:47| Accepting ICP messages at 0.0.0.0,
> > > > port 3130,
> > > > >> > FD 122. 2009/11/03 18:59:47| Accepting HTCP messages on port
> > > > 4827, FD
> > > > >> > 123. 2009/11/03 18:59:47| Accepting SNMP messages on port 3401,
> > > > FD 124.
> > > > >> > 2009/11/03 18:59:47| Pinger socket opened on FD 126
> > > > >> > 2009/11/03 18:59:47| Configuring Parent 127.0.0.1/8080/7
> > > > >> > 2009/11/03 18:59:47| Configuring Parent 10.10.50.232/8080/7
> > > > >> >
> > > > >> > I wonder to know if there is a way i can speed this up.
> > > > >>
> > > > >> An old rule-of-thumb in computing is to start with the first
> > > > reported
> > > > >> problem and see how many of the following disappearr
> > > > >>
> > > > >> For you that is as Squid suggests "Please check your
> > > > /etc/resolv.conf
> > > > >>  file".
> > > > >>
> > > > >> The slow server seems to have no DNS servers available. This
> > > > could be
> > > > >> causing any kind of problems for the helpers later on. All of the
> > > > extra
> > > > >> delay is during the startup process of the helpers.
> > > > >>
> > > > >> Amos
> > > > >>
> > > > >
> > > > > You were right, I was missing DNS, whoever, look, delay is still
> > > > slow. about
> > > > > 54 sec to reconfigure.  What other think I may change.  This is a
> > > > loaded
> > > > > server, with about 1k connections.
> > > > >
> > > > > 2009/11/04 11:15:35| Reconfiguring Squid Cache (version
> > > > 3.0.STABLE19)...
> > > > > 2009/11/04 11:15:35| FD 76 Closing HTTP connection
> > > > > 2009/11/04 11:15:35| FD 141 Closing ICP connection
> > > > > 2009/11/04 11:15:35| FD 142 Closing HTCP socket
> > > > > 2009/11/04 11:15:35| FD 143 Closing SNMP socket
> > > > > 2009/11/04 11:15:35| Processing Configuration File:
> > > > /etc/squid/squid.conf
> > > > > (depth 0)
> > > > > 2009/11/04 11:15:35| Processing Configuration File:
> > > > > /etc/squid/squid.acl (depth
> > > > > 1)
> > > > > 2009/11/04 11:15:35| WARNING: HTTP requires the use of Via
> > > > > 2009/11/04 11:15:35| Initializing https proxy context
> > > > > 2009/11/04 11:15:35| Store logging disabled
> > > > > 2009/11/04 11:15:35| User-Agent logging is disabled.
> > > > > 2009/11/04 11:15:35| Referer logging is disabled.
> > > > > 2009/11/04 11:15:35| DNS Socket created at 0.0.0.0, port 60779, FD 8
> > > > > 2009/11/04 11:15:35| Adding nameserver 127.0.0.1 from squid.conf
> > > > > 2009/11/04 11:15:35| helperOpenServers: Starting 32/32
> > > > 'squidGuard' processes
> > > > > 2009/11/04 11:15:51| helperOpenServers: Starting 16/16
> > > > 'digest_ldap_auth'
> > > > > processes
> > > > > 2009/11/04 11:15:59| helperOpenServers: Starting 32/32
> > > > 'squid_ldap_group'
> > > > > processes
> > > > > 2009/11/04 11:16:15| helperOpenServers: Starting 32/32
> > > > 'ip-user-helper.pl'
> > > > > processes
> > > > > 2009/11/04 11:16:31| Accepting  HTTP connections at 0.0.0.0, port
> > > > > 3128, FD 78.
> > > > > 2009/11/04 11:16:31| Accepting ICP messages at 0.0.0.0, port 3130,
> > > > FD 143.
> > > > > 2009/11/04 11:16:31| Accepting HTCP messages on port 4827, FD 144.
> > > > > 2009/11/04 11:16:31| Accepting SNMP messages on port 3401, FD 391.
> > > > >
> > >
> > >
> > > --
> > > ---------------------------------------
> > > Malte Schröder
> > > MalteSch@xxxxxx
> > > ICQ# 68121508
> > > ---------------------------------------
>
>
> --
> Jorge Armando Medina
> Computación Gráfica de México
> Web: http://www.e-compugraf.com
> Tel: 55 51 40 72, Ext: 124
> Email: jmedina@xxxxxxxxxxxxxxx
> GPG Key: 1024D/28E40632 2007-07-26
> GPG Fingerprint: 59E2 0C7C F128 B550 B3A6  D3AF C574 8422 28E4 0632