On Mon, Nov 23, 2009 at 8:06 PM, Amos Jeffries <squid3@xxxxxxxxxxxxx> wrote: > Accepting the challenge and going of the side tangent ;) .... > > redirect helpers can be reduced or removed in a lot of cases by using: Our redirectors aren't the problem. They work fine for 23:55 every day; over 10 billion served. :-) It's solely the overhead of fork()ing 16 copies of the squid executable and its mammoth address space to start new ones when the logs are rotated. The redirectors do a lot of dynamic heavy lifting, and there's simply no way to replace them with a static configuration. I watched this happen today, because I noticed we accidentally had the 24GB machine with cache_mem set to 12GB. I tried to cut it back to 6GB with a squid -k reconfig. At the time, the squid process was right at 24GB VSZ, with about 15GB RSS. It started spawning child processes very, very slowly, but it was the *parent* process that went insane with CPU usage... it pegged its CPU at 100% and the load average on the machine went to about 6. The spawned child processes had the same VSZ but a tiny little RSS, about like I guess one would expect. I don't know if it's just in the kernel copying the processes' page tables for the next fork(), or if there's something else going on during reconfig that would cause that kind of CPU usage. But it seems to stall out everything. > Maybe. We would like to diagnose this problem and fix it properly, but if > its too much hassle you can go that way. It would definitely be my preference to diagnose and fix the problem and I can live with a fair amount of hassle to get there. (Unless you are saying "you are using redirectors" is the problem, in which case memfs it is. ;-) ) Thanks!
