On 29/05/2013 2:19 p.m., Nathan Hoad wrote:
On Tue, May 28, 2013 at 3:23 PM, Amos Jeffries <squid3@xxxxxxxxxxxxx> wrote:
On 28/05/2013 3:59 p.m., Nathan Hoad wrote:
I take it you are referring to the 2.0g resident size?
That is what I'm referring to, yes - the resident size has increased
to 2.5g since my previous mail, virtual to 2.6g.
1GB is within the reasonable use limit for a fully loaded Squid under peak
traffic. The resident size reported is usually the biggest "ever" size of
memory usage by the process.
FWIW: The memory report shows about 324MB being tracked by Squid as
currently in use for other things than cache_mem with 550 active clients
doing 117 transactions at present. The client transaction related pools show
that the current values are 1/3 of peak traffic, so 3x 360MB ==> ~1GB under
peak traffic appears entirely possible for your Squid.
Out of interest, how did you come to the 324MB? I'd be interested in
learning how to read the output a bit better :)
Okay. (for anyone reading the report is a TSV format [open in Libre Calc
or Excel as Tab-separated columns]).
The final row of the big table is Totals of all rows above. I took the
1360314 from Allocated section "(KB)" column [~1360 MB] and subtracted
1GB / 1024MB. That is the current Total memory usage Squid is aware of
either in active use or waiting re-allocation, minus what you said
cache_mem was configured to.
NP: Before subtracting I did a quick check of the mem_node (ie cache_mem
memory 'pages') there is ~1025 MB. Enough for the full 1024 MB cache_mem
and some extra MB of items in-transit right now that are not cacheable -
which use mem_nodes as well.
Using a bit of developer inside-knowledge I identify in the Allocated
section "(#)" column the main state objects which are allocated one for
each client connection:
* "cbdata ConnStateData" shows 2749 - at 1 per currently open client TCP
connection.
* "cbdata ClientHttpRequest " shows 550 - at 1 each per client HTTP request.
++ sorry I got that wrong earlier myself.
* "cbdata ClientReplyContext" also shows 550 - at 1 each per currently
underway client HTTP responses.
These objects also give me the details to estimate current versus peak
traffic memory requirements. For example:
cbdata ClientReplyContext shows 550 current allocated, using 35269 KB,
with a highest-ever allocation of 100933 KB - roughy 3x the current
memory usage.
The difference between Allocated "(KB)" column and "high (KB)" columns
shows us how much is allocated now versus the highest ever allocated. A
leak usually shows up as both those columns being nearly identical
values, although it is possible that 1:10 objects leaks or something
weird like that which can hide it.
I may have to change my reading a bit. Thinking about that column
meaning now I see the Total value for allocations "high (KB)" only shows
1462831 KB - its not very accurate, but does show that if all objects
were reaching their max at the same time it would still be ~500MB short
of that 2GB the system reports. Another option that usually adds fuzz to
these numbers is spawning of helpers - the fork() used for that
allocates the child process a whole duplicate of the current memory
space usage of the parent process. Effectively doubling the OS-reported
memory values from whatever the reality is.
Like Alex said, leaks show up as an ever-increasing value in these
numbers somewhere. Regular snapshots of that report and the system
values taken across a week or two should be able to show if there is
anything constantly growing at a regular rate.
Amos
