On 09/06/11 20:16, Le Trung Kien wrote:
Hi, I don't know how to handle memory for squid. Squid automatically breaks down itself and restarts:
http://wiki.squid-cache.org/SquidFaq/SquidMemory
In my squid.conf I set: url_rewrite_children 120 startup=30 idle=100 concurrency=0
You need 3.2 for the dynamic startup capabilities. 3.1 only uses the first value (120 helpers to start *immediately*).
Given that is actually a reasonable number of helpers I think the memory consumption elsewhere is probably killing them.
max_filedesc 7168 cache_swap_low 60 cache_swap_high 80
If you are forced to do that with swap values it means you cache management is badly calculated.
There is no use allocating for example 100GB of disk storage when cache_swap_low forces 40% of it not to be used. And Squid pausing occasionally while 20% of it (80%-60%) is erased can also cause visible slowdown to clients.
The memory FAQ page linked above outlines how to estimate cache sizes.
I notice that just one pending requests queued: 2011/06/09 11:00:17| WARNING: All redirector processes are busy. 2011/06/09 11:00:17| WARNING: 1 pending requests queued and follow is more details of cache.log: 2011/06/09 11:00:16| WARNING: Cannot run '/opt/squid-3.1.10/urlrewriter_new.pl' process. 2011/06/09 11:00:16| ipcCreate: fork: (12) Cannot allocate memory
2011/06/09 11:00:16| WARNING: Cannot run '/opt/squid-3.1.10/urlrewriter_new.pl' process. 2011/06/09 11:00:16| ipcCreate: fork: (12) Cannot allocate memory 2011/06/09 11:00:16| WARNING: Cannot run '/opt/squid-3.1.10/urlrewriter_new.pl' process. 2011/06/09 11:00:16| ipcCreate: fork: (12) Cannot allocate memory 2011/06/09 11:00:16| WARNING: Cannot run '/opt/squid-3.1.10/urlrewriter_new.pl' process. 2011/06/09 11:00:17| WARNING: All redirector processes are busy. 2011/06/09 11:00:17| WARNING: 1 pending requests queued 2011/06/09 11:00:17| storeDirWriteCleanLogs: Starting... 2011/06/09 11:00:17| WARNING: Closing open FD 177 2011/06/09 11:00:17| Finished. Wrote 16367 entries. 2011/06/09 11:00:17| Took 0.01 seconds (1308941.14 entries/sec). FATAL: Too many queued redirector requests Squid Cache (Version 3.1.10): Terminated abnormally.
Your machine seems not to be capable of running 120 or these rewriter processes. You need more available to them RAM or less maximum helpers.
<---------------------------------- CRASH *** CPU Usage: 6631.678 seconds = 4839.266 user + 1792.412 sys Maximum Resident Size: 0 KB Page faults with physical i/o: 3299
"Page faults" - your system is swapping badly to get that many just on startup. If this continues it will kill Squid performance later even if it gets past the helpers.
Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.12 Beta testers wanted for 3.2.0.8 and 3.1.12.2
