Hi, I decreased the number of url_rewrite_children and upgraded squid to version 3.1.12, now after two days in weekend (Sat and Sun) the memory consuming has not exceeded the amount of defined cache_mem and it seems to be working properly :) Page faults with physical i/o: 0 I still monitoring it on working days to see what happen, but I thinks the problem may be caused by too much redirector programs spawning. Thank you. Kien Le. On Fri, Jun 10, 2011 at 1:27 PM, Amos Jeffries <squid3@xxxxxxxxxxxxx> wrote: > On 09/06/11 20:16, Le Trung Kien wrote: >> >> Hi, I don't know how to handle memory for squid. Squid automatically >> breaks down itself and restarts: > > http://wiki.squid-cache.org/SquidFaq/SquidMemory > >> >> In my squid.conf I set: >> >> url_rewrite_children 120 startup=30 idle=100 concurrency=0 > > You need 3.2 for the dynamic startup capabilities. 3.1 only uses the first > value (120 helpers to start *immediately*). > > Given that is actually a reasonable number of helpers I think the memory > consumption elsewhere is probably killing them. > > > >> max_filedesc 7168 >> cache_swap_low 60 >> cache_swap_high 80 > > If you are forced to do that with swap values it means you cache management > is badly calculated. > > There is no use allocating for example 100GB of disk storage when > cache_swap_low forces 40% of it not to be used. And Squid pausing > occasionally while 20% of it (80%-60%) is erased can also cause visible > slowdown to clients. > > The memory FAQ page linked above outlines how to estimate cache sizes. > >> >> I notice that just one pending requests queued: >> >> 2011/06/09 11:00:17| WARNING: All redirector processes are busy. >> 2011/06/09 11:00:17| WARNING: 1 pending requests queued >> >> and follow is more details of cache.log: >> >> 2011/06/09 11:00:16| WARNING: Cannot run >> '/opt/squid-3.1.10/urlrewriter_new.pl' process. >> 2011/06/09 11:00:16| ipcCreate: fork: (12) Cannot allocate memory > > >> 2011/06/09 11:00:16| WARNING: Cannot run >> '/opt/squid-3.1.10/urlrewriter_new.pl' process. >> 2011/06/09 11:00:16| ipcCreate: fork: (12) Cannot allocate memory >> 2011/06/09 11:00:16| WARNING: Cannot run >> '/opt/squid-3.1.10/urlrewriter_new.pl' process. >> 2011/06/09 11:00:16| ipcCreate: fork: (12) Cannot allocate memory >> 2011/06/09 11:00:16| WARNING: Cannot run >> '/opt/squid-3.1.10/urlrewriter_new.pl' process. >> 2011/06/09 11:00:17| WARNING: All redirector processes are busy. >> 2011/06/09 11:00:17| WARNING: 1 pending requests queued >> 2011/06/09 11:00:17| storeDirWriteCleanLogs: Starting... >> 2011/06/09 11:00:17| WARNING: Closing open FD 177 >> 2011/06/09 11:00:17| Finished. Wrote 16367 entries. >> 2011/06/09 11:00:17| Took 0.01 seconds (1308941.14 entries/sec). >> FATAL: Too many queued redirector requests >> Squid Cache (Version 3.1.10): Terminated abnormally. > > Your machine seems not to be capable of running 120 or these rewriter > processes. You need more available to them RAM or less maximum helpers. > > >> <---------------------------------- CRASH *** >> CPU Usage: 6631.678 seconds = 4839.266 user + 1792.412 sys >> Maximum Resident Size: 0 KB >> Page faults with physical i/o: 3299 > > "Page faults" - your system is swapping badly to get that many just on > startup. If this continues it will kill Squid performance later even if it > gets past the helpers. > > Amos > -- > Please be using > Current Stable Squid 2.7.STABLE9 or 3.1.12 > Beta testers wanted for 3.2.0.8 and 3.1.12.2 >
