Search squid archive

Re: squid: Memory utilization higher than expected since moving from 3.3 to 3.4 and Vary: working

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 07/17/2014 02:49 AM, Martin Sperl wrote:
> This is why I have been mentioning all the pools that show similar (wavy)
> memory increase pattern. There must be one of those that is the root of 
> all the others.

Unfortunately, I do not share your certainty. As I see it, at least two
theories more-or-less fit your data:

A) MemObjects are leaking.

B) Your memory cache is growing, explaining some or all of the pools
growth and muddying the waters along the way. Something unrelated to
MemObjects is leaking. That something may or may not be pooled.

My bet is on (B).

If you do not mind reminding me, does accounted-for memory pools growth
correspond to the actual/total Squid memory growth? Or is the "pooled"
vs "total/real" gap widening with time?


> the strange thing is that if you look at the
> distribution of vm_objects, then you see that they have expired long ago
> (16267 days ago to be exact, so with EX:-1 - 42511 exactly).
> If these have been expired, then why would they get loaded into memory? 

because you have free cache space and Squid can still serve that cached
data (after revalidation)? In general, there is no benefit from purging
something from a non-full cache, especially if that something is reusable.

I have not checked the code and responses producing those "16267 days"
stats so I treat that number with a grain of salt. :-)


> Well: SUM(inmem_hi) is memory for payload (possibly without headers) against
> which we compare the cache_mem against.
> 
> But if the squid process consumes 9GB, then there must be more than a factor 
> of 2 of overhead so that we get to those 9GB.

Yes if (A) above is correct. If (B) above is correct, that "overhead" is
a leak of something other than memory cache entries.


> Report-file(with Date+Time) psmemory MemObjects kb/MemObject
> report-20140708-234225.txt    1795932     123979     6.03
...
> report-20140717-000001.txt   10662148     845404    11.37

> Which shows that the average size/memObject is increasing constantly!

Yes, and that is one of the reasons I bet on (B). (B) is a lot less
surprising than a constantly growing MemObject overhead that (A) has to
explain :-). Running with a small cache size (i.e., with a full cache)
would confirm that.



> Another point here more on the reporting/debugging side:
>         Total space in arena:  -2034044 KB

Yes, this is a known problem mostly outside of Squid control:
http://www.squid-cache.org/mail-archive/squid-users/201402/0296.html

Squid should probably stop using that API [when it can detect that it is
broken].


And yes, I know that my response does not really help you. I just wanted
to make sure you consider both (A) and (B) theories in your investigation.


Alex.
P.S. Another suggestion that you probably cannot use in your environment
is to run trunk or a trunk-based branch. Perhaps the leak you are after
has been fixed. And if not, somebody may be more motivated to help you
find it in trunk.





[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux