Mr Dash Four wrote:
Alex Rousskov wrote:
On 04/10/2013 09:22 AM, Mr Dash Four wrote:
the total memory used of all objects placed in ram should
not be 6 times the cache_mem value
There is no such rule or law of physics. cache_mem is not squid_mem (a
not-yet-supported option that would limit total Squid memory usage).
The total memory used by Squid is, roughly:
1) initial memory used by Squid shortly after start +
2) memory used to store memory-cached objects +
3a) memory used to index memory-cached objects +
3b) memory used to index disk-cached objects +
4) memory used for current transactions +
5) memory leaked since Squid start
cache_mem determines the maximum value of (2)
cache_mem and mean object size in memory cache determine the maximum
value of (3a) because they determine the maximum number of objects in
the memory cache.
cache_mem has no significant direct effect on the other four components.
Depending on the local circumstances, every other component alone may
exceed cache_mem 5 times or more, especially when cache_mem is
relatively small, e.g., 200MB. And taken together, they can exceed
6*cache_mem level even easier.
For example, a 10KB/s memory leak will consume 1GB of RAM in less than
two days. Similarly, an overloaded Squid may grow (4) at rates exceeding
1MB/s.
Well, in that case, whoever did the testing of squid before releasing
such a buggy software should be shot on site or be locked in a dark
garage 24/7 somewhere and taught how to use valgrind.
Now back to your actual problem. An idle Squid with 200 MB memory cache
and 200 MB disk cache should not use 1.3GB of RAM unless component (1)
is very high due to some huge ACL lists or other unusual complications.
Nope. I have 5 rules, that's all.
If your (1) is just a few MBs and your idle Squid uses 1.3GB, then I
would suspect a memory leak.
At start, squid memory consumption is about 25-30MB - give or take a
few megabytes, keeping in mind that according to the logs I have in
excess of 14500 objects (verified by squid at startup) in the disk cache.
If you want to reduce Squid memory usage, I recommend eliminating memory
leaks as suspects first because they are bugs and because they may
consume all available RAM on your machine.
I am not a squid developer - I am a user.
Whether your Squid leaks
memory, I do not know. I do know that you are running v3.2.0 that
probably does have leaks and a set of other nasty bugs fixed in
currently supported Squid versions.
I'll try to see if the latest version has the above behaviour and if
that turns out to be the case and I have found no solution the the
problem I described in my initial post, then I have to just forget
that squid exists and use something else which is more reliable, like
polipo for example.
I do thank you for your time and your comments though - much appreciated.
I am keeping the entire post to which I reply, because it has been quite
a while since I posted in this thread.
Just to recap - I was using squid 3.2.0 and had massive memory problems
with my system (atom-based machine with 2GB RAM) due to squid
cannibalising my entire free memory pool, using in excess of 1.3GB of it
(keeping in mind that I have "restricted" squid to use 200MB, not more
than 6 times that amount).
After a period of discussing various issues on this thread, I decided to
upgrade squid to 3.3.4 and see if that cures the problem. Well, it did!
My squid memory usage currently stands at little over 104MB and that is
after nearly 2 months of continues use, bearing in mind that with the
previous version (3.2.0) that memory was in excess of 1.3GB in just over
a month of such use. I did not post these results earlier as I wanted to
be sure that this isn't some sort of fluke and also that the new squid
is capable of running for much longer than the the old one, while
keeping the memory use in check.
It needs to be noted that there is no (significant) differences in the
configuration between the old and the new versions, apart from minor
corrections to account for changed option names (transition from 3.2.0
to 3.3.4). There isn't any difference in the usage (nature or intensity)
either. Apart from a new kernel, the only difference is the new squid
version.
Conclusion: squid 3.2.0 should be confined to the dust bin!
I've read an advisory which recommends upgrading to 3.3.7 (and now 3.3.8
is released), so I'll upgrade to that in the coming days, but wanted to
post this update in the meantime for the benefit of this group.
