Search squid archive

Re: Squid uses way too much RAM and starts swapping ...

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 11/05/11 19:19, guest01 wrote:
Hi,

I am currently using squid 3.1.12 as forward-proxy without
harddisk-caching (only RAM is used for caching). Each server is
running on RHEL5.5 and is pretty strong (16 CPUs, 28GB RAM), but each
servers starts swapping a few days after start. The workaround at the
moment is to reboot the server once a week, which I don't really like.
But swapping leads to serious side effects, e.g. performance troubles,
...

way too much swapping:
http://imageshack.us/m/52/6149/memoryday.png

I already read a lot of posts and mails for similar problems, but
unfortunately, I was not able to solve this problem. I added following
infos to my squid.conf-file:
# cache specific settings
cache_replacement_policy heap LFUDA
cache_mem 1600 MB
memory_replacement_policy heap LFUDA
maximum_object_size_in_memory 2048 KB
memory_pools off
cache_swap_low 85
cache_swap_high 90

(There are four squid instances per server, which means that 1600*4 =
6400MB RAM used for caching, which is not even 1/4 of the total
available amount of RAM. Plenty enough, don't you think?)

Not that is "for HTTP object caching", emphasis on *caching* and "HTTP object". In-transit objects and non-HTTP caches (Ip cache, domain name cache, persistent connections cache, client database, via/fwd database, network performance cache, auth caches, external ACL caches) and the indexes for all those caches use other memory.

Then again they should all be using no more than a few GB combined. So you may have hit a new leak (all the known ones are resolved before 3.1.12).


Very strange are the negative values (Memory usage for squid via
mallinfo():) from the output below. Maybe that is a reason for running
out of RAM?

mallinfo() sucks badly when going above 2GB of RAM. It can be ignored.

The section underneath it "Memory accounted for:" is Squids own accounting and more of a worry. It should not have had negatives since before 3.1.10.


HTTP/1.0 200 OK
Server: squid/3.1.12
Mime-Version: 1.0
Date: Wed, 11 May 2011 07:06:10 GMT
Content-Type: text/plain
Expires: Wed, 11 May 2011 07:06:10 GMT
Last-Modified: Wed, 11 May 2011 07:06:10 GMT
X-Cache: MISS from xlsqip03_1
Via: 1.0 xlsqip03_1 (squid/3.1.12)
Connection: close

Squid Object Cache: Version 3.1.12
Start Time:     Wed, 27 Apr 2011 11:01:13 GMT
Current Time:   Wed, 11 May 2011 07:06:10 GMT
Connection information for squid:
         Number of clients accessing cache:      1671
         Number of HTTP requests received:       16144359
         Number of ICP messages received:        0
         Number of ICP messages sent:    0
         Number of queued ICP replies:   0
         Number of HTCP messages received:       0
         Number of HTCP messages sent:   0
         Request failure ratio:   0.00
         Average HTTP requests per minute since start:   810.3
         Average ICP messages per minute since start:    0.0
         Select loop called: 656944758 times, 1.820 ms avg
Cache information for squid:
         Hits as % of all requests:      5min: 17.4%, 60min: 18.2%
         Hits as % of bytes sent:        5min: 45.6%, 60min: 39.9%
         Memory hits as % of hit requests:       5min: 86.1%, 60min: 88.9%
         Disk hits as % of hit requests: 5min: 0.0%, 60min: 0.0%
         Storage Swap size:      0 KB
         Storage Swap capacity:   0.0% used,  0.0% free
         Storage Mem size:       1622584 KB
         Storage Mem capacity:   100.0% used,  0.0% free

Okay 1.6 GB of RAM used for caching HTTP objects. Fully used.

         Mean Object Size:       0.00 KB

Problem #1. It *may* be Squid not accounting for the memory objects in the mean.

         Requests given to unlinkd:      0
Median Service Times (seconds)  5 min    60 min:
         HTTP Requests (All):   0.01648  0.01235
         Cache Misses:          0.05046  0.04277
         Cache Hits:            0.00091  0.00091
         Near Hits:             0.01469  0.01745
         Not-Modified Replies:  0.00000  0.00091
         DNS Lookups:           0.00190  0.00190
         ICP Queries:           0.00000  0.00000
Resource usage for squid:
         UP Time:        1195497.286 seconds
         CPU Time:       22472.507 seconds
         CPU Usage:      1.88%
         CPU Usage, 5 minute avg:        5.38%
         CPU Usage, 60 minute avg:       5.44%
         Process Data Segment Size via sbrk(): 3145032 KB
         Maximum Resident Size: 0 KB
         Page faults with physical i/o: 8634
Memory usage for squid via mallinfo():
         Total space in arena:  -1049140 KB
         Ordinary blocks:       -1277813 KB  87831 blks
         Small blocks:               0 KB      0 blks
         Holding blocks:          2240 KB      5 blks
         Free Small blocks:          0 KB
         Free Ordinary blocks:  228673 KB
         Total in use:          -1275574 KB 122%
         Total free:            228674 KB -22%
         Total size:            -1046900 KB
Memory accounted for:
         Total accounted:       -1375357 KB 131%
         memPool accounted:     2818947 KB -269%
         memPool unaccounted:   -3865847 KB 0%
         memPoolAlloc calls:       111
         memPoolFree calls:  8322084644
File descriptor usage for squid:
         Maximum number of file descriptors:   1024
         Largest file desc currently in use:    563
         Number of file desc currently in use:  472
         Files queued for open:                   0
         Available number of file descriptors:  552
         Reserved number of file descriptors:   100
         Store Disk files open:                   0
Internal Data Structures:
          96996 StoreEntries
          96996 StoreEntries with MemObjects
          96980 Hot Object Cache Items
              0 on-disk objects

Has anyone experienced similar things or does even know a solution?

Not since we fixed Squid's capacity to calculate storage used by >2GB objects.

Can you find the actual maximum and average object size for the cached objects in that Squid? (the mgr:vm_objects report should have all their details, "inmem_hi:" being the object size)

Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.12
  Beta testers wanted for 3.2.0.7 and 3.1.12.1


[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux