Hi,
I don't have any scientific metrics regarding mem cache versus disk
and cache, apart from metrics collected by keynote.com which show a
slight inprovment in overall site speed which may or may not be
related.. I had been using both memory and disk until fairly recently
but feel I should have enough RAM to cache frequently accessed files
in Memory only and avoid any potential disk buffer issues during
flashmob type events where our traffic can potentially increase 10
fold in a matter of minutes.
Will try LRU on one of the machines next time I do a restart.
squidclient output below.
HTTP/1.0 200 OK
Server: squid
Date: Tue, 27 Nov 2007 16:33:36 GMT
Content-Type: text/plain
Expires: Tue, 27 Nov 2007 16:33:36 GMT
Last-Modified: Tue, 27 Nov 2007 16:33:36 GMT
X-Cache: MISS from www.xxxxxxx.xxx
X-Cache-Lookup: MISS from www.xxxxxxx.xxx:80
Via: 1.0 www.xxxxxxxx.xxx:80 (squid)
Connection: close
Squid Object Cache: Version 2.6.STABLE6
Start Time: Mon, 26 Nov 2007 11:16:17 GMT
Current Time: Tue, 27 Nov 2007 16:33:36 GMT
Connection information for squid:
Number of clients accessing cache: 43833
Number of HTTP requests received: 14361302
Number of ICP messages received: 0
Number of ICP messages sent: 0
Number of queued ICP replies: 0
Request failure ratio: 0.00
Average HTTP requests per minute since start: 8172.3
Average ICP messages per minute since start: 0.0
Select loop called: 175178637 times, 0.602 ms avg
Cache information for squid:
Request Hit Ratios: 5min: 87.5%, 60min: 87.2%
Byte Hit Ratios: 5min: 48.2%, 60min: 53.3%
Request Memory Hit Ratios: 5min: 44.7%, 60min: 45.0%
Request Disk Hit Ratios: 5min: 0.1%, 60min: 0.2%
Storage Swap size: 0 KB
Storage Mem size: 6181676 KB
Mean Object Size: 0.00 KB
Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
HTTP Requests (All): 0.00091 0.00179
Cache Misses: 0.00286 0.00286
Cache Hits: 0.00179 0.00179
Near Hits: 0.00562 0.00678
Not-Modified Replies: 0.00091 0.00091
DNS Lookups: 0.00000 0.03223
ICP Queries: 0.00000 0.00000
Resource usage for squid:
UP Time: 105439.283 seconds
CPU Time: 5072.935 seconds
CPU Usage: 4.81%
CPU Usage, 5 minute avg: 8.48%
CPU Usage, 60 minute avg: 7.95%
Process Data Segment Size via sbrk(): 8582112 KB
Maximum Resident Size: 0 KB
Page faults with physical i/o: 0
Memory usage for squid via mallinfo():
Total space in arena: 193504 KB
Ordinary blocks: 193413 KB 14 blks
Small blocks: 0 KB 0 blks
Holding blocks: 17104 KB 3 blks
Free Small blocks: 0 KB
Free Ordinary blocks: 90 KB
Total in use: 210517 KB 100%
Total free: 90 KB 0%
Total size: 210608 KB
Memory accounted for:
Total accounted: 7735377 KB
memPoolAlloc calls: 1284461973
memPoolFree calls: 1253790650
File descriptor usage for squid:
Maximum number of file descriptors: 16384
Largest file desc currently in use: 4287
Number of file desc currently in use: 4080
Files queued for open: 0
Available number of file descriptors: 12304
Reserved number of file descriptors: 100
Store Disk files open: 0
IO loop method: epoll
Internal Data Structures:
1138371 StoreEntries
1138371 StoreEntries with MemObjects
1138143 Hot Object Cache Items
0 on-disk objects
J
On Nov 26, 2007 2:55 PM, Tek Bahadur Limbu <teklimbu@xxxxxxxxxxxx> wrote:
> Hi John,
>
> John Moylan wrote:
> > Hi,
> >
> > I have three memory only caches set up 7GB of memory each (the
> > machines have 12GB of physical memory each). Throughput is fairly high
> > and this setup works well in reducing the number of requests for
> > smaller files from my backend storage with lower latency that a disk
> > and mem. solution.
>
> Do you have statistics regarding fetching from memory and disk? How much
> is the performance increment when using memory cache only?
>
>
> However, the cache's on of the machines fill up
> > every 2-3 days and Squid's CPU usage subsequently goes up to 100%
> > (These are all dual SMP machines and system load average remains
> > around 0.7). FD's, the number of connections and swap are all fine
> > when the CPU goes up so the culprit is more than likely to be cache
> > replacement.
> >
> > I am using heap GDSF as the policy. The maximum size in memory is set
> > to 96 KB.
>
> Have you tried the LFUDA or the default LRU memory replacement policies?
>
> I am using squid-2.6.STABLE6-4.el5 on Linux 2.6.
>
> Try upgrading to the latest version of squid.
>
> http://www.squid-cache.org/Versions/v2/2.6/squid-2.6.STABLE16.tar.gz
>
> It probably contains some improvements over version 2.6.6.
>
> >
> > Is there anything I can do to improve expensive cache replacement
> > apart from stopping and starting Squid every day?
>
> By the way, which Linux distro are you using?
>
> Can you post the output of "squidclient mgr:info" or the relevant parts
> of your squid.conf?
>
> Thanking you...
>
>
> >
> > J
> >
> >
> >
>
>
> --
>
> With best regards and good wishes,
>
> Yours sincerely,
>
> Tek Bahadur Limbu
>
> System Administrator
>
> (TAG/TDG Group)
> Jwl Systems Department
>
> Worldlink Communications Pvt. Ltd.
>
> Jawalakhel, Nepal
>
> http://www.wlink.com.np
>
> http://teklimbu.wordpress.com
>
