On Wed, 28 Jul 2010 18:13:03 +0200, Ralf Hildebrandt <Ralf.Hildebrandt@xxxxxxxxxx> wrote: > * Murilo Moreira de Oliveira <murilo.moreira@xxxxxxxxx>: >> Hi guys. I'm using squid 3.1.4 and since of first 3.1 stable release I'm >> suffering from memory leak problems too. My production server is a CentOS >> 5.4 32 bits (kernel 2.6.18-164.6.1.el5). It has 2GB of RAM (512MB >> reserved >> to squid cache_mem) and 6GB of disk space reserved to squid's disk >> cache. In >> approximately 24h, squid consumes all the available memory of my server >> and >> swap starts to be consumed. I'm currently restarting squid once a day in >> order to workaround the problem. Follow attached a squidclient output >> made >> some hours before all the server memory be consumed. > > Today I performed some more tests and completely disabled delay_pools > in 3.1.5 and checked if that may fix the excessive memory usage > > After a short while, memory use skyrocketed - again. > > One interessant observation I made is that squid-rrd/1day.cgi reports a > sudden spike in "Cached Objects -- 1 day" > > With 3.0.x, the number of objects cached in memory was more or less > constant (see green line), once 3.1.5 took over, the numbers skyrocket > (peak at 1.8M, meaning it's actually only 18.000 items) maximum_object_in_memory default was increased a bit in 3.1 to encompass most of the bell curve of small 'average' objects. 3.0 and earlier only memory-cached the lower half of the range. Unless you explicitly configured cache_mem its default size has also taken a bigger jump to accommodate ~10,000 of the average memory cacheable objects. 3.1 will now also do true promotion of objects from disk into memory cache if they become hot. Two other changes in 3.1 related to HTTP/1.1 which have since been shown to indirectly affect the hot objects: * There are two confirmed major content providers whose server software sends out different cacheability headers based on whether the client is HTTP/1.0 or HTTP/1.1. They seem to let 1.1 clients cache more than 1.0 clients. * during the 1.0->1.1 upgrade of replies Squid synthesizes a Date: header which increases the cacheability of many responses up from zero. > > The image can be found here: http://www.arschkrebs.de/bugs/squid-object.png > Access forbidden apparently. > This image had been created using poll.pl: > > # Originally at: > # http://wessels.squid-cache.org/squid-rrd/ > # Rewritten by Dan Kogai <dankogai@xxxxxxxxx> > # Rewritten by Duane Wessels <wessels@xxxxxxxxxxxxxxx> > # poll.pl,v 1.9 2004/03/30 18:55:27 wessels Exp > > According to the source, these (the green graph) are created from the > "hot" items: > > $Vals{nhot} = $1 if (/(\d+)\s+Hot Object Cache Items/i); > > > objects => { > DS => { > disk => { format => '%d', type => 'GAUGE', valkey => > 'nobj', }, > mem => { format => '%d', type => 'GAUGE', valkey => 'nhot', }, Amos
