Search squid archive

Re: cache size and structure

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



My cache traffic volume (I/O) is about 2 Mbps a week with peaks of 3 Mbps.

ehm, 2 megabits per socond "a week"?

excuse me, 1 megabit per second for a week, so :
1 Mb *60*60*24*7=604.800 / 8 = 75.600 MB  about 76 GB traffic for week.
I'm refering both inbound and outbound traffic (1 Mbps in , 1 Mbps out).


If users acess your squid directly (not only through child proxies), there
may be the need for cache_mem.

some users access by child squids


I have "48 256" for 20GiB cache. If you take the number of files in the
cache directory, divide by 256 (l2 dirs) and 256 (max files in l2 dir), you
should get the approximate need of l1 dirs. The average object size is
aropund 13KiB, which means, you should have one L1 cache_Dir per ~800MiB of
cache size. Splitting small files to COSS (using min-size option for *ufs
and max-size for COSS) will make that even smaller number, since only big
iles will be placed to *ufs directory hierarchy.


I'm sorry but I didn't understand.
How can I enable COSS ?


----- Original Message ----- From: "Matus UHLAR - fantomas" <uhlar@xxxxxxxxxxx>
To: <squid-users@xxxxxxxxxxxxxxx>
Sent: Thursday, June 25, 2009 11:02 AM
Subject: Re:  cache size and structure


Riccardo Castellani wrote:
I'm preparing new squid machine and I'm defining cache size.
Old squid had 2 entries into "cache_dir directive" :

cache_dir ufs /usr/local/cache/1 3500 128 256
cache_dir ufs /usr/local/cache/2 2500 128 256

On 23.06.09 13:07, Chris Robertson wrote:
I'd strongly suggest using "aufs" instead of "ufs".

seconded.

My cache traffic volume (I/O) is about 2 Mbps a week with peaks of 3 Mbps.

ehm, 2 megabits per socond "a week"?

This squid cache is the parent of 2 other squid machines and it gives
answers to about 1000 users.

1- I read you suggest 1 cache_dir to same partition, why I can't use 2
folder in tha same partition ?!

You can, but why would you want to?  The suggestion is one cache_dir per
spindle to spread the IO load.  Putting multiple partitions on one
spindle makes about the same sense as multiple cache_dirs in the same
partition.  Access to all of them will be contending for the limited IO
resources available.

That means, different cache_dirs are mostly for using different disks.
Or different storage schema, e.g. using COSS (which is perfect for small
files) in addition to *ufs

2- What do you think my caches size ? 3500 and 2550 ?

Depends on your memory load.  A larger cache leads to storing more
objects, which requires more memory to track.  The suggestion I recall
is "a week's worth of traffic".  If you are seeing an average of 2Mbit/s
24 hours a day, seven days a week, that would lead to a cache of around
150GB.

I'd say it depends on disk load also :) caching one-two weeks of HTTP
traffic is not that good if disk becomes the leak. But switching to one
aufs cache_dir (plus optional COSS) should give enough of speed benefits so
the cache could be increased.

If users acess your squid directly (not only through child proxies), there
may be the need for cache_mem.

See please squid FAQ for informacions about memory usage...
http://wiki.squid-cache.org/SquidFaq/SquidMemory

 and its directory structure (128,256) ?!

This is entirely dependent on the filesystem you are using and the
number of objects you cache.  The goal is to keep the number of files
per directory reasonable, because most filesystems are not optimized for
a "large" ratio (10s of thousands of files per directory).

I have "48 256" for 20GiB cache. If you take the number of files in the
cache directory, divide by 256 (l2 dirs) and 256 (max files in l2 dir), you
should get the approximate need of l1 dirs. The average object size is
aropund 13KiB, which means, you should have one L1 cache_Dir per ~800MiB of
cache size. Splitting small files to COSS (using min-size option for *ufs
and max-size for COSS) will make that even smaller number, since only big
iles will be placed to *ufs directory hierarchy.

--
Matus UHLAR - fantomas, uhlar@xxxxxxxxxxx ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Emacs is a complicated operating system without good text editor.


[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux