squid 3.5.23 memory usage

Ivan Larionov <xeron.oskom@xxxxxxxxx> · Thu, 19 Jan 2017 16:23:54 -0800

Hello.
I'm pretty sure this question has been asked multiple times already, but after reading everything I found I still can't figure out squid memory usage patterns.

We're currently trying to upgrade from squid 2.7 to squid 3.5 and memory usage on squid 3 is much much higher compared to squid 2 with the same configuration.

What do I see:

squid running for several days with low traffic:

# top
 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 7367 squid     20   0 4780m 4.4g 5224 S  6.0 60.6 105:01.76 squid -N

So it uses 4.4GB resident memory. Ok, let's see important config options:

cache_mem 2298756 KB
maximum_object_size_in_memory 8 KB
memory_replacement_policy lru
cache_replacement_policy lru

cache_dir aufs /mnt/services/squid/cache 445644 16 256

minimum_object_size 64 bytes # none-zero so we dont cache mistakes
maximum_object_size 102400 KB

So we configured 2.2GB memory cache and 500GB disk cache. Disk cache is quite big but current usage is only 3GB:

# du -sh /mnt/services/squid/cache # cache_dir
3.0G  /mnt/services/squid/cache

Now I'm looking into this page http://wiki.squid-cache.org/SquidFaq/SquidMemory and see:

14 MB of memory per 1 GB on disk for 64-bit Squid

Which means disk cache should use ~50MB of RAM.

All these means we have ~2.2GB ram used for everything else except cache_mem and disk cache index.

Let's see top pools from mgr:mem:

Pool                  (KB)     %Tot
mem_node              2298833  55.082
Short Strings         622365   14.913
HttpHeaderEntry       404531   9.693
Long Strings          284520   6.817
MemObject             182288   4.368
HttpReply             155612   3.729
StoreEntry            73965    1.772
Medium Strings        71152    1.705
cbdata MemBuf (12)    35573    0.852
LRU policy node       30403    0.728
MD5 digest            11380    0.273
16K Buffer            1056     0.025

These pools consume ~35% of total squid memory usage: Short Strings, HttpHeaderEntry, Long Strings, HttpReply. Looks suspicious. On squid 2 same pools use 10 times less memory.

I found a bug which looks similar to our experience: http://bugs.squid-cache.org/show_bug.cgi?id=4084.

I'm attaching our config, mgr:info, mgr:mem and some system info I collected.

Could someone say if this is normal and why it's so much different from squid 2?

-- 
With best regards, Ivan Larionov.

HTTP/1.1 200 OK
Server: squid/3.5.23
Mime-Version: 1.0
Date: Thu, 19 Jan 2017 23:39:50 GMT
Content-Type: text/plain;charset=utf-8
Expires: Thu, 19 Jan 2017 23:39:50 GMT
Last-Modified: Thu, 19 Jan 2017 23:39:50 GMT
X-Cache: MISS from ip-172-22-10-120
X-Cache-Lookup: MISS from ip-172-22-10-120:3128
Connection: close

Squid Object Cache: Version 3.5.23
Build Info: 
Service Name: squid
Start Time:	Fri, 13 Jan 2017 23:35:32 GMT
Current Time:	Thu, 19 Jan 2017 23:39:50 GMT
Connection information for squid:
	Number of clients accessing cache:	(client_db off)
	Number of HTTP requests received:	8195690
	Number of ICP messages received:	0
	Number of ICP messages sent:	0
	Number of queued ICP replies:	0
	Number of HTCP messages received:	0
	Number of HTCP messages sent:	0
	Request failure ratio:	 0.00
	Average HTTP requests per minute since start:	948.1
	Average ICP messages per minute since start:	0.0
	Select loop called: 73529108 times, 7.054 ms avg
Cache information for squid:
	Hits as % of all requests:	5min: 29.2%, 60min: 28.9%
	Hits as % of bytes sent:	5min: 89.0%, 60min: 89.1%
	Memory hits as % of hit requests:	5min: 0.0%, 60min: 0.0%
	Disk hits as % of hit requests:	5min: 100.0%, 60min: 100.0%
	Storage Swap size:	2915344 KB
	Storage Swap capacity:	 0.6% used, 99.4% free
	Storage Mem size:	2276524 KB
	Storage Mem capacity:	99.0% used,  1.0% free
	Mean Object Size:	4.00 KB
	Requests given to unlinkd:	0
Median Service Times (seconds)  5 min    60 min:
	HTTP Requests (All):   0.01745  0.01745
	Cache Misses:          0.02899  0.02451
	Cache Hits:            0.00091  0.00091
	Near Hits:             0.00000  0.00000
	Not-Modified Replies:  0.00000  0.00000
	DNS Lookups:           0.00000  0.00094
	ICP Queries:           0.00000  0.00000
Resource usage for squid:
	UP Time:	518657.265 seconds
	CPU Time:	6265.444 seconds
	CPU Usage:	1.21%
	CPU Usage, 5 minute avg:	6.43%
	CPU Usage, 60 minute avg:	5.11%
	Maximum Resident Size: 18579360 KB
	Page faults with physical i/o: 0
Memory accounted for:
	Total accounted:       -20826 KB
	memPoolAlloc calls: 2192400061
	memPoolFree calls:  2194290230
File descriptor usage for squid:
	Maximum number of file descriptors:   524288
	Largest file desc currently in use:     70
	Number of file desc currently in use:   44
	Files queued for open:                   0
	Available number of file descriptors: 524244
	Reserved number of file descriptors:   100
	Store Disk files open:                   0
Internal Data Structures:
	728426 StoreEntries
	569027 StoreEntries with MemObjects
	569025 Hot Object Cache Items
	728372 on-disk objects

HTTP/1.1 200 OK
Server: squid/3.5.23
Mime-Version: 1.0
Date: Thu, 19 Jan 2017 23:39:38 GMT
Content-Type: text/plain;charset=utf-8
Expires: Thu, 19 Jan 2017 23:39:38 GMT
Last-Modified: Thu, 19 Jan 2017 23:39:38 GMT
X-Cache: MISS from ip-172-22-10-120
X-Cache-Lookup: MISS from ip-172-22-10-120:3128
Connection: close

Current memory usage:
Pool  Obj Size Chunks Allocated In Use Idle Allocations Saved Rate
  (bytes) KB/ch  obj/ch (#)  used  free  part  %Frag  (#)  (KB)  high (KB)  high (hrs)  %Tot (#)  (KB)  high (KB)  high (hrs)  %alloc (#)  (KB)  high (KB) (#)  %cnt  %vol (#)/sec
mem_node             4136         569150  2298833  2299059  4.14  55.082  569132  2298760  2299059  4.14  99.997  18  73  1596  6403998  0.288  4.225  30.618
Short Strings          40         15932521  622365  622443  25.27  14.913  15932253  622354  622443  25.27  99.998  268  11  268  1286674648  57.866  8.209  6921.017
HttpHeaderEntry        56         7397138  404531  404577  25.27  9.693  7397028  404525  404577  25.27  99.999  110  7  180  333424345  14.995  2.978  1835.071
Long Strings          512         569039  284520  284525  23.43  6.817  568971  284486  284525  23.43  99.988  68  34  161  4379781  0.197  0.358  36.678
MemObject             328         569092  182288  182290  25.27  4.368  569028  182267  182290  25.27  99.989  64  21  101  7584864  0.341  0.397  42.083
HttpReply             280         569093  155612  155614  25.27  3.729  569029  155594  155614  25.27  99.989  64  18  86  22742497  1.023  1.016  115.072
StoreEntry            104         728265  73965  73965  0.00  1.772  728265  73965  73965  0.00  100.000  0  0  5  6234621  0.280  0.103  30.575
Medium Strings        128         569213  71152  71152  0.58  1.705  569035  71130  71152  0.58  99.969  178  23  51  49767099  2.238  1.016  284.894
cbdata MemBuf (12)     64         569160  35573  35573  23.43  0.852  569031  35565  35573  23.43  99.977  129  9  24  37847992  1.702  0.386  187.961
LRU policy node        24         1297183  30403  30403  0.00  0.728  1297183  30403  30403  0.00  100.000  0  0  8  4811841  0.216  0.018  8.027
MD5 digest             16         728265  11380  11380  0.00  0.273  728265  11380  11380  0.00  100.000  0  0  1  10518494  0.473  0.027  37.535
16K Buffer           16384         66  1056  1056  0.58  0.025  2  32  1056  0.58  3.030  64  1024  1056  6912649  0.311  18.065  30.478
cbdata idns_query (8)  8696         38  323  323  77.63  0.008  0  0  323  77.63  0.000  38  323  323  35939  0.002  0.050  0.080
cbdata clientReplyContext (18)  4320         67  283  283  0.58  0.007  3  13  283  0.58  4.478  64  270  283  8194685  0.369  5.647  42.414
cbdata ClientSocketContext (17)  4256         67  279  279  0.58  0.007  3  13  279  0.58  4.478  64  266  279  8194685  0.369  5.563  42.414
16KB Strings         16384         15  240  240  79.77  0.006  1  16  240  79.77  6.667  14  224  240  8194744  0.369  21.415  42.415
2K Buffer            2048         67  134  134  0.58  0.003  6  12  134  0.58  8.955  61  122  128  47412503  2.132  15.488  247.468
HttpRequest          1784         67  117  117  0.58  0.003  3  6  117  0.58  4.478  64  112  117  8194881  0.369  2.332  42.415
8K Buffer            8192         6  48  48  103.86  0.001  0  0  48  103.86  0.000  6  48  48  323805  0.015  0.423  5.928
Comm::Connection      200         208  41  41  0.58  0.001  38  8  41  0.58  18.269  170  34  38  25729544  1.157  0.821  126.886
squidaio_small_bufs  4096         9  36  36  103.86  0.001  0  0  36  103.86  0.000  9  36  36  6391602  0.287  4.176  54.896
cbdata Server (15)    416         67  28  28  0.58  0.001  3  2  28  0.58  4.478  64  26  28  8194692  0.369  0.544  42.415
cbdata ClientHttpRequest (16)   384         67  26  26  0.58  0.001  3  2  26  0.58  4.478  64  24  26  8194685  0.369  0.502  42.414
MimeEntry             128         177  23  23  144.07  0.001  177  23  23  144.07  100.000  0  0  0  0  0.000  0.000  0.000
cbdata HttpStateData (28)   320         66  21  21  0.58  0.000  2  1  21  0.58  3.030  64  20  21  6910805  0.311  0.353  30.475
4K Buffer            4096         5  20  20  51.01  0.000  0  0  20  51.01  0.000  5  20  20  909966  0.041  0.595  5.918
fqdncache_entry       160         124  20  20  77.63  0.000  104  17  20  77.63  83.871  20  4  6  35926  0.002  0.001  0.080
cbdata clientStreamNode (19)   128         134  17  17  0.58  0.000  6  1  17  0.58  4.478  128  16  17  16389370  0.737  0.335  84.829
4KB Strings          4096         4  16  16  2.19  0.000  0  0  16  2.19  0.000  4  16  16  2188  0.000  0.001  0.215
cbdata FwdState (25)   176         66  12  12  0.58  0.000  2  1  12  0.58  3.030  64  11  12  6910805  0.311  0.194  30.475
1KB Strings          1024         11  11  11  52.21  0.000  1  1  11  52.21  9.091  10  10  10  259578  0.012  0.042  0.610
cbdata store_client (22)   160         67  11  11  0.58  0.000  3  1  11  0.58  4.478  64  10  11  10661246  0.479  0.272  42.414
ipcache_entry         128         75  10  10  144.07  0.000  75  10  10  144.07  100.000  0  0  0  0  0.000  0.000  0.000
cbdata BodyPipe (24)   152         62  10  10  0.58  0.000  2  1  10  0.58  3.226  60  9  10  3710839  0.167  0.090  23.519
cbdata IdleConnList (29)  4160         2  9  9  143.97  0.000  2  9  9  143.97  100.000  0  0  9  26033  0.001  0.017  0.189
HttpHdrCc              96         71  7  7  133.18  0.000  53  5  7  133.18  74.648  18  2  2  14644821  0.659  0.224  56.489
MemBlob                48         123  6  6  0.58  0.000  58  3  6  0.58  47.154  65  4  4  55415101  2.492  0.424  255.046
Acl::AndNode          136         26  4  4  144.07  0.000  26  4  4  144.07  100.000  0  0  0  0  0.000  0.000  0.000
cbdata Tree (3)       192         10  2  2  144.07  0.000  10  2  2  144.07  100.000  0  0  0  0  0.000  0.000  0.000
cbdata CachePeer (4)   872         2  2  2  144.07  0.000  2  2  2  144.07  100.000  0  0  0  0  0.000  0.000  0.000
cbdata UFSStoreState (31)   176         9  2  2  103.86  0.000  0  0  2  103.86  0.000  9  2  2  4421615  0.199  0.124  18.785
squidaio_micro_bufs   128         12  2  2  13.07  0.000  0  0  2  13.07  0.000  12  2  2  6882960  0.310  0.141  18.785
aio_ctrl              112         12  2  2  13.07  0.000  0  0  2  13.07  0.000  12  2  2  13274571  0.597  0.237  73.681
acl_ip_data            96         13  2  2  144.07  0.000  13  2  2  144.07  100.000  0  0  1  4  0.000  0.000  0.000
aio_request           104         12  2  2  13.07  0.000  0  0  2  13.07  0.000  12  2  2  13274571  0.597  0.220  73.681
cbdata generic_cbdata (23)    32         38  2  2  77.63  0.000  0  0  2  77.63  0.000  38  2  2  3225497  0.145  0.016  7.026
cbdata ConnOpener (27)   136         8  2  2  124.86  0.000  0  0  2  124.86  0.000  8  2  2  3713239  0.167  0.081  23.522
cbdata ACLFilledChecklist (21)   456         2  1  1  144.04  0.000  1  1  1  144.04  50.000  1  1  1  15105621  0.679  1.099  72.889
cbdata DiskThreadsDiskFile (32)    88         9  1  1  103.86  0.000  0  0  1  103.86  0.000  9  1  1  4421615  0.199  0.062  18.785
cbdata RebuildState (11)   688         1  1  1  144.07  0.000  0  0  1  144.07  0.000  1  1  1  0  0.000  0.000  0.000
cbdata ReadRequest (35)    72         9  1  1  103.86  0.000  0  0  1  103.86  0.000  9  1  1  6391602  0.287  0.073  54.896
aio_thread             40         16  1  1  144.04  0.000  16  1  1  144.04  100.000  0  0  0  0  0.000  0.000  0.000
ACLSourceIP           120         5  1  1  144.07  0.000  5  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
ev_entry               48         12  1  1  144.07  0.000  11  1  1  144.07  91.667  1  1  1  731619  0.033  0.006  1.439
cbdata CbDataList (1)    40         11  1  1  144.07  0.000  11  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
cbdata IoResult (36)    40         9  1  1  103.86  0.000  0  0  1  103.86  0.000  9  1  1  6391602  0.287  0.041  54.896
RegexList              88         4  1  1  144.07  0.000  4  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
cbdata Logfile (10)   352         1  1  1  144.07  0.000  1  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
ACLStringData          56         6  1  1  144.07  0.000  6  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
ACLRegexData           16         20  1  1  144.07  0.000  20  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
UFSStoreState::_queued_write    32         10  1  12  4.14  0.000  0  0  12  4.14  0.000  10  1  12  6379196  0.287  0.033  13.893
ACLStrategised        136         2  1  1  144.07  0.000  2  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
ACLStrategised        136         2  1  1  144.07  0.000  2  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
ACLStrategised        136         2  1  1  144.07  0.000  2  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
cbdata ps_state (26)   256         1  1  1  144.04  0.000  0  0  1  144.04  0.000  1  1  1  6910870  0.311  0.282  30.475
UFSStoreState::_queued_read    40         6  1  1  103.86  0.000  0  0  1  103.86  0.000  6  1  1  1232060  0.055  0.008  11.840
cbdata RemovalPolicy (5)   104         2  1  1  144.07  0.000  2  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
cbdata StoreSearchHashIndex (14)   104         2  1  1  144.07  0.000  1  1  1  144.07  50.000  1  1  1  144  0.000  0.000  0.000
ACLUserData            64         3  1  1  144.07  0.000  3  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
ACLHTTPHeaderData      48         4  1  1  144.07  0.000  4  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
link_list              16         10  1  6  4.14  0.000  0  0  6  4.14  0.000  10  1  6  7611262  0.342  0.019  25.733
ACLStrategised        136         1  1  1  144.07  0.000  1  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
Acl::NotNode          136         1  1  1  144.07  0.000  1  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
cbdata CbDataList (2)    64         2  1  1  144.07  0.000  2  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
ACLDestinationIP      120         1  1  1  144.07  0.000  1  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
cbdata TcpAcceptor (13)   104         1  1  1  144.07  0.000  1  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
cbdata ClientRequestContext (20)   104         1  1  1  144.07  0.000  0  0  1  144.07  0.000  1  1  1  8194751  0.369  0.136  42.414
cbdata CbDataList (30)    96         1  1  1  144.04  0.000  0  0  1  144.04  0.000  1  1  1  275  0.000  0.000  0.001
cbdata WriteRequest (33)    80         1  1  1  144.04  0.000  0  0  1  144.04  0.000  1  1  1  6379579  0.287  0.081  13.893
StoreSwapLogData       72         1  1  1  144.04  0.000  0  0  1  144.04  0.000  1  1  1  5594760  0.252  0.064  6.946
cbdata RemovalPurgeWalker (38)    72         1  1  1  25.30  0.000  0  0  1  25.30  0.000  1  1  1  14055  0.001  0.000  0.515
wordlist               16         4  1  1  144.07  0.000  0  0  1  144.07  0.000  4  1  1  79  0.000  0.000  0.000
cbdata RemovalPolicyWalker (37)    56         1  1  1  132.04  0.000  0  0  1  132.04  0.000  1  1  1  5  0.000  0.000  0.000
ACLMethodData          16         3  1  1  144.07  0.000  3  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
dwrite_q               48         1  1  1  144.07  0.000  0  0  1  144.07  0.000  1  1  1  11974341  0.539  0.092  20.839
cbdata AclSizeLimit (6)    48         1  1  1  144.07  0.000  1  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
FwdServer              24         2  1  1  144.04  0.000  0  0  1  144.04  0.000  2  1  1  13821740  0.622  0.053  60.949
ACLNoteData            40         1  1  1  144.07  0.000  1  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
CacheDigest            40         1  1  1  144.07  0.000  1  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
cbdata IoResult (34)    40         1  1  1  144.04  0.000  0  0  1  144.04  0.000  1  1  1  6379579  0.287  0.041  13.893
ACLDomainData          16         2  1  1  144.07  0.000  2  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
ACLHierCodeData        32         1  1  1  144.07  0.000  1  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
ACLTimeData            32         1  1  1  144.07  0.000  1  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
ACLASN                 16         2  1  1  144.07  0.000  2  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
StoreMetaMD5           32         1  1  1  144.04  0.000  0  0  1  144.04  0.000  1  1  1  4393969  0.198  0.022  18.540
StoreMetaSTDLFS        32         1  1  1  144.04  0.000  0  0  1  144.04  0.000  1  1  1  4393969  0.198  0.022  18.540
StoreMetaURL           32         1  1  1  144.04  0.000  0  0  1  144.04  0.000  1  1  1  4393969  0.198  0.022  18.540
StoreMetaObjSize       32         1  1  1  144.04  0.000  0  0  1  144.04  0.000  1  1  1  4393969  0.198  0.022  18.540
StoreMetaVary          32         1  1  1  143.47  0.000  0  0  1  143.47  0.000  1  1  1  0  0.000  0.000  0.000
ACLProtocolData        16         1  1  1  144.07  0.000  1  1  1  144.07  100.000  0  0  0  0  0.000  0.000  0.000
Total                   1         29500179  4173438  4173478  0.00  100.000  29497928  4170607  4170837  0.05  99.932  2251  2832  4425  -2147483648  98.588  99.306  368715.124
Cumulative allocated volume: 626.942 GB
Current overhead: 36331 bytes (0.001%)
Idle pool limit: 5.00 MB
Total Pools created: 116
Pools ever used:     100 (shown above)
Currently in use:    66
String Pool  Impact
  (%strings)  (%volume)
Short Strings        93  64
Medium Strings       3  7
Long Strings         3  29
1KB Strings          0  0
4KB Strings          0  0
16KB Strings         0  0
Other Strings        0  0

Large buffers: 0 (0 KB)

Attachment:
squid.conf

Description: Binary data
# free -m

             total       used       free     shared    buffers     cached
Mem:          7482       7349        133          0        189       1354
-/+ buffers/cache:       5805       1677
Swap:            0          0          0

# df -h

Filesystem                   Size  Used Avail Use% Mounted on
/dev/xvda1                   9.8G  3.3G  6.4G  34% /
devtmpfs                     3.7G   68K  3.7G   1% /dev
tmpfs                        3.7G   44K  3.7G   1% /dev/shm
/dev/mapper/vg-data--master   29G  1.1G   28G   4% /mnt
/dev/xvdh                    504G  3.0G  476G   1% /mnt/services/squid/cache

# du -sh /mnt/services/squid/cache

3.0G  /mnt/services/squid/cache

# top

 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 7367 squid     20   0 4780m 4.4g 5224 S  6.0 60.6 105:01.76 squid -N

# squid -v
Squid Cache: Version 3.5.23
Service Name: squid
configure options:  '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/sbin' '--sbindir=/usr/sbin' '--sysconfdir=/etc/squid' '--libdir=/usr/lib' '--libexecdir=/usr/lib/squid' '--includedir=/usr/include' '--datadir=/usr/share' '--sharedstatedir=/usr/com' '--localstatedir=/var' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--enable-epoll' '--enable-removal-policies=heap,lru' '--enable-storeio=aufs' '--enable-delay-pools' '--with-pthreads' '--enable-cache-digests' '--enable-useragent-log' '--enable-referer-log' '--with-large-files' '--with-maxfd=16384' '--enable-err-languages=English' '--enable-htcp'
_______________________________________________
squid-users mailing list
squid-users@xxxxxxxxxxxxxxxxxxxxx
http://lists.squid-cache.org/listinfo/squid-users

HTTP/1.1 200 OK
Server: squid/3.5.23
Mime-Version: 1.0
Date: Thu, 19 Jan 2017 23:39:38 GMT
Content-Type: text/plain;charset=utf-8
Expires: Thu, 19 Jan 2017 23:39:38 GMT
Last-Modified: Thu, 19 Jan 2017 23:39:38 GMT
X-Cache: MISS from ip-172-22-10-120
X-Cache-Lookup: MISS from ip-172-22-10-120:3128
Connection: close
Current memory usage:
Pool	Obj Size	Chunks							Allocated					In Use					Idle			Allocations Saved			Rate
	(bytes)	KB/ch	obj/ch	(#)	used	free	part	%Frag	(#)	(KB)	high (KB)	high (hrs)	%Tot	(#)	(KB)	high (KB)	high (hrs)	%alloc	(#)	(KB)	high (KB)	(#)	%cnt	%vol	(#)/sec
mem_node	4136								569150	2298833	2299059	4.14	55.082	569132	2298760	2299059	4.14	99.997	18	73	1596	6403998	0.288	4.225	30.618
Short Strings	40								15932521	622365	622443	25.27	14.913	15932253	622354	622443	25.27	99.998	268	11	268	1286674648	57.866	8.209	6921.017
HttpHeaderEntry	56								7397138	404531	404577	25.27	9.693	7397028	404525	404577	25.27	99.999	110	7	180	333424345	14.995	2.978	1835.071
Long Strings	512								569039	284520	284525	23.43	6.817	568971	284486	284525	23.43	99.988	68	34	161	4379781	0.197	0.358	36.678
MemObject	328								569092	182288	182290	25.27	4.368	569028	182267	182290	25.27	99.989	64	21	101	7584864	0.341	0.397	42.083
HttpReply	280								569093	155612	155614	25.27	3.729	569029	155594	155614	25.27	99.989	64	18	86	22742497	1.023	1.016	115.072
StoreEntry	104								728265	73965	73965	0.00	1.772	728265	73965	73965	0.00	100.000	0	0	5	6234621	0.280	0.103	30.575
Medium Strings	128								569213	71152	71152	0.58	1.705	569035	71130	71152	0.58	99.969	178	23	51	49767099	2.238	1.016	284.894
cbdata MemBuf (12)	64								569160	35573	35573	23.43	0.852	569031	35565	35573	23.43	99.977	129	9	24	37847992	1.702	0.386	187.961
LRU policy node	24								1297183	30403	30403	0.00	0.728	1297183	30403	30403	0.00	100.000	0	0	8	4811841	0.216	0.018	8.027
MD5 digest	16								728265	11380	11380	0.00	0.273	728265	11380	11380	0.00	100.000	0	0	1	10518494	0.473	0.027	37.535
16K Buffer	16384								66	1056	1056	0.58	0.025	2	32	1056	0.58	3.030	64	1024	1056	6912649	0.311	18.065	30.478
cbdata idns_query (8)	8696								38	323	323	77.63	0.008	0	0	323	77.63	0.000	38	323	323	35939	0.002	0.050	0.080
cbdata clientReplyContext (18)	4320								67	283	283	0.58	0.007	3	13	283	0.58	4.478	64	270	283	8194685	0.369	5.647	42.414
cbdata ClientSocketContext (17)	4256								67	279	279	0.58	0.007	3	13	279	0.58	4.478	64	266	279	8194685	0.369	5.563	42.414
16KB Strings	16384								15	240	240	79.77	0.006	1	16	240	79.77	6.667	14	224	240	8194744	0.369	21.415	42.415
2K Buffer	2048								67	134	134	0.58	0.003	6	12	134	0.58	8.955	61	122	128	47412503	2.132	15.488	247.468
HttpRequest	1784								67	117	117	0.58	0.003	3	6	117	0.58	4.478	64	112	117	8194881	0.369	2.332	42.415
8K Buffer	8192								6	48	48	103.86	0.001	0	0	48	103.86	0.000	6	48	48	323805	0.015	0.423	5.928
Comm::Connection	200								208	41	41	0.58	0.001	38	8	41	0.58	18.269	170	34	38	25729544	1.157	0.821	126.886
squidaio_small_bufs	4096								9	36	36	103.86	0.001	0	0	36	103.86	0.000	9	36	36	6391602	0.287	4.176	54.896
cbdata Server (15)	416								67	28	28	0.58	0.001	3	2	28	0.58	4.478	64	26	28	8194692	0.369	0.544	42.415
cbdata ClientHttpRequest (16)	384								67	26	26	0.58	0.001	3	2	26	0.58	4.478	64	24	26	8194685	0.369	0.502	42.414
MimeEntry	128								177	23	23	144.07	0.001	177	23	23	144.07	100.000	0	0	0	0	0.000	0.000	0.000
cbdata HttpStateData (28)	320								66	21	21	0.58	0.000	2	1	21	0.58	3.030	64	20	21	6910805	0.311	0.353	30.475
4K Buffer	4096								5	20	20	51.01	0.000	0	0	20	51.01	0.000	5	20	20	909966	0.041	0.595	5.918
fqdncache_entry	160								124	20	20	77.63	0.000	104	17	20	77.63	83.871	20	4	6	35926	0.002	0.001	0.080
cbdata clientStreamNode (19)	128								134	17	17	0.58	0.000	6	1	17	0.58	4.478	128	16	17	16389370	0.737	0.335	84.829
4KB Strings	4096								4	16	16	2.19	0.000	0	0	16	2.19	0.000	4	16	16	2188	0.000	0.001	0.215
cbdata FwdState (25)	176								66	12	12	0.58	0.000	2	1	12	0.58	3.030	64	11	12	6910805	0.311	0.194	30.475
1KB Strings	1024								11	11	11	52.21	0.000	1	1	11	52.21	9.091	10	10	10	259578	0.012	0.042	0.610
cbdata store_client (22)	160								67	11	11	0.58	0.000	3	1	11	0.58	4.478	64	10	11	10661246	0.479	0.272	42.414
ipcache_entry	128								75	10	10	144.07	0.000	75	10	10	144.07	100.000	0	0	0	0	0.000	0.000	0.000
cbdata BodyPipe (24)	152								62	10	10	0.58	0.000	2	1	10	0.58	3.226	60	9	10	3710839	0.167	0.090	23.519
cbdata IdleConnList (29)	4160								2	9	9	143.97	0.000	2	9	9	143.97	100.000	0	0	9	26033	0.001	0.017	0.189
HttpHdrCc	96								71	7	7	133.18	0.000	53	5	7	133.18	74.648	18	2	2	14644821	0.659	0.224	56.489
MemBlob	48								123	6	6	0.58	0.000	58	3	6	0.58	47.154	65	4	4	55415101	2.492	0.424	255.046
Acl::AndNode	136								26	4	4	144.07	0.000	26	4	4	144.07	100.000	0	0	0	0	0.000	0.000	0.000
cbdata Tree (3)	192								10	2	2	144.07	0.000	10	2	2	144.07	100.000	0	0	0	0	0.000	0.000	0.000
cbdata CachePeer (4)	872								2	2	2	144.07	0.000	2	2	2	144.07	100.000	0	0	0	0	0.000	0.000	0.000
cbdata UFSStoreState (31)	176								9	2	2	103.86	0.000	0	0	2	103.86	0.000	9	2	2	4421615	0.199	0.124	18.785
squidaio_micro_bufs	128								12	2	2	13.07	0.000	0	0	2	13.07	0.000	12	2	2	6882960	0.310	0.141	18.785
aio_ctrl	112								12	2	2	13.07	0.000	0	0	2	13.07	0.000	12	2	2	13274571	0.597	0.237	73.681
acl_ip_data	96								13	2	2	144.07	0.000	13	2	2	144.07	100.000	0	0	1	4	0.000	0.000	0.000
aio_request	104								12	2	2	13.07	0.000	0	0	2	13.07	0.000	12	2	2	13274571	0.597	0.220	73.681
cbdata generic_cbdata (23)	32								38	2	2	77.63	0.000	0	0	2	77.63	0.000	38	2	2	3225497	0.145	0.016	7.026
cbdata ConnOpener (27)	136								8	2	2	124.86	0.000	0	0	2	124.86	0.000	8	2	2	3713239	0.167	0.081	23.522
cbdata ACLFilledChecklist (21)	456								2	1	1	144.04	0.000	1	1	1	144.04	50.000	1	1	1	15105621	0.679	1.099	72.889
cbdata DiskThreadsDiskFile (32)	88								9	1	1	103.86	0.000	0	0	1	103.86	0.000	9	1	1	4421615	0.199	0.062	18.785
cbdata RebuildState (11)	688								1	1	1	144.07	0.000	0	0	1	144.07	0.000	1	1	1	0	0.000	0.000	0.000
cbdata ReadRequest (35)	72								9	1	1	103.86	0.000	0	0	1	103.86	0.000	9	1	1	6391602	0.287	0.073	54.896
aio_thread	40								16	1	1	144.04	0.000	16	1	1	144.04	100.000	0	0	0	0	0.000	0.000	0.000
ACLSourceIP	120								5	1	1	144.07	0.000	5	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
ev_entry	48								12	1	1	144.07	0.000	11	1	1	144.07	91.667	1	1	1	731619	0.033	0.006	1.439
cbdata CbDataList (1)	40								11	1	1	144.07	0.000	11	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
cbdata IoResult (36)	40								9	1	1	103.86	0.000	0	0	1	103.86	0.000	9	1	1	6391602	0.287	0.041	54.896
RegexList	88								4	1	1	144.07	0.000	4	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
cbdata Logfile (10)	352								1	1	1	144.07	0.000	1	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
ACLStringData	56								6	1	1	144.07	0.000	6	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
ACLRegexData	16								20	1	1	144.07	0.000	20	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
UFSStoreState::_queued_write	32								10	1	12	4.14	0.000	0	0	12	4.14	0.000	10	1	12	6379196	0.287	0.033	13.893
ACLStrategised	136								2	1	1	144.07	0.000	2	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
ACLStrategised	136								2	1	1	144.07	0.000	2	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
ACLStrategised	136								2	1	1	144.07	0.000	2	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
cbdata ps_state (26)	256								1	1	1	144.04	0.000	0	0	1	144.04	0.000	1	1	1	6910870	0.311	0.282	30.475
UFSStoreState::_queued_read	40								6	1	1	103.86	0.000	0	0	1	103.86	0.000	6	1	1	1232060	0.055	0.008	11.840
cbdata RemovalPolicy (5)	104								2	1	1	144.07	0.000	2	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
cbdata StoreSearchHashIndex (14)	104								2	1	1	144.07	0.000	1	1	1	144.07	50.000	1	1	1	144	0.000	0.000	0.000
ACLUserData	64								3	1	1	144.07	0.000	3	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
ACLHTTPHeaderData	48								4	1	1	144.07	0.000	4	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
link_list	16								10	1	6	4.14	0.000	0	0	6	4.14	0.000	10	1	6	7611262	0.342	0.019	25.733
ACLStrategised	136								1	1	1	144.07	0.000	1	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
Acl::NotNode	136								1	1	1	144.07	0.000	1	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
cbdata CbDataList (2)	64								2	1	1	144.07	0.000	2	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
ACLDestinationIP	120								1	1	1	144.07	0.000	1	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
cbdata TcpAcceptor (13)	104								1	1	1	144.07	0.000	1	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
cbdata ClientRequestContext (20)	104								1	1	1	144.07	0.000	0	0	1	144.07	0.000	1	1	1	8194751	0.369	0.136	42.414
cbdata CbDataList (30)	96								1	1	1	144.04	0.000	0	0	1	144.04	0.000	1	1	1	275	0.000	0.000	0.001
cbdata WriteRequest (33)	80								1	1	1	144.04	0.000	0	0	1	144.04	0.000	1	1	1	6379579	0.287	0.081	13.893
StoreSwapLogData	72								1	1	1	144.04	0.000	0	0	1	144.04	0.000	1	1	1	5594760	0.252	0.064	6.946
cbdata RemovalPurgeWalker (38)	72								1	1	1	25.30	0.000	0	0	1	25.30	0.000	1	1	1	14055	0.001	0.000	0.515
wordlist	16								4	1	1	144.07	0.000	0	0	1	144.07	0.000	4	1	1	79	0.000	0.000	0.000
cbdata RemovalPolicyWalker (37)	56								1	1	1	132.04	0.000	0	0	1	132.04	0.000	1	1	1	5	0.000	0.000	0.000
ACLMethodData	16								3	1	1	144.07	0.000	3	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
dwrite_q	48								1	1	1	144.07	0.000	0	0	1	144.07	0.000	1	1	1	11974341	0.539	0.092	20.839
cbdata AclSizeLimit (6)	48								1	1	1	144.07	0.000	1	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
FwdServer	24								2	1	1	144.04	0.000	0	0	1	144.04	0.000	2	1	1	13821740	0.622	0.053	60.949
ACLNoteData	40								1	1	1	144.07	0.000	1	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
CacheDigest	40								1	1	1	144.07	0.000	1	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
cbdata IoResult (34)	40								1	1	1	144.04	0.000	0	0	1	144.04	0.000	1	1	1	6379579	0.287	0.041	13.893
ACLDomainData	16								2	1	1	144.07	0.000	2	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
ACLHierCodeData	32								1	1	1	144.07	0.000	1	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
ACLTimeData	32								1	1	1	144.07	0.000	1	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
ACLASN	16								2	1	1	144.07	0.000	2	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
StoreMetaMD5	32								1	1	1	144.04	0.000	0	0	1	144.04	0.000	1	1	1	4393969	0.198	0.022	18.540
StoreMetaSTDLFS	32								1	1	1	144.04	0.000	0	0	1	144.04	0.000	1	1	1	4393969	0.198	0.022	18.540
StoreMetaURL	32								1	1	1	144.04	0.000	0	0	1	144.04	0.000	1	1	1	4393969	0.198	0.022	18.540
StoreMetaObjSize	32								1	1	1	144.04	0.000	0	0	1	144.04	0.000	1	1	1	4393969	0.198	0.022	18.540
StoreMetaVary	32								1	1	1	143.47	0.000	0	0	1	143.47	0.000	1	1	1	0	0.000	0.000	0.000
ACLProtocolData	16								1	1	1	144.07	0.000	1	1	1	144.07	100.000	0	0	0	0	0.000	0.000	0.000
Total	1								29500179	4173438	4173478	0.00	100.000	29497928	4170607	4170837	0.05	99.932	2251	2832	4425	-2147483648	98.588	99.306	368715.124
Cumulative allocated volume: 626.942 GB
Current overhead: 36331 bytes (0.001%)
Idle pool limit: 5.00 MB
Total Pools created: 116
Pools ever used: 100 (shown above)
Currently in use: 66
String Pool	Impact
	(%strings)	(%volume)
Short Strings	93	64
Medium Strings	3	7
Long Strings	3	29
1KB Strings	0	0
4KB Strings	0	0
16KB Strings	0	0
Other Strings	0	0
Large buffers: 0 (0 KB)