Youtube may be hogging your pipe but it is better to know than to guess.
The access.log shows content sizes so with a simple awk script it should
be easy to find out.
I have also seen many sites where advertisements and trackers consume 15%
bandwidth. This may vary. So blocking ads and trackers is a thing to
consider.
Do not expect too much from web caching. More and more websites intentionally
make their sites not cacheable. Look at the percentage of TCP_MISS in
access.log or use a second awk script to find out more about cacheability.
First some information about the setup: the hardware itself is a Xeon E3110
server with 8 GB of RAM and lots of diskspace. OS is CentOS 6.2, a derivate of
Red Hat Enterprise Linux and I'm using the CentOS flavour of Squid, version
squid-3.1.10-1.el6_2.2.x86_64.
I recommend going for a newer Squid: 3.1.19 is stable and fixes issues that
3.1.10 has.
Half a TB is planned for squid webobjects with the following line:
cache_dir diskd /var/cache/proxy/squid 512000 16 256 Q1=72 Q2=64
On Linux, aufs has a better performance than diskd
Additional memory for storing objects is 2048 MB:
cache_mem 2048 MB
Seems right. But you also need virtual memory for Squid being able to
fork processes without issues. Do have have 8 GB swap ?
But read the FAQ about memory usage and a large disk cache:
http://wiki.squid-cache.org/SquidFaq/SquidMemory
Squid uses an additional 512*14 MB = 7.1 GB for the index of the disk cache.
I suggest to downsize to 1 GB in-memory index which implies to
use only 73 GB disk cache.
Squid works in combination with an NGINX proxy setup for caching youtube video
content, as this is probably the greatest bandwith hog. It is configured as a
cache_peer and a regexp acl:
acl youtube_videos url_regex -i ^http://[^/]+(\.youtube\.com|\.googlevideo\.com|\.video\.google\.com)/(videoplayback|get_video|videodownload)\?
acl range_request req_header Range .
acl begin_param url_regex -i [?&]begin=
acl id_param url_regex -i [?&]id=
acl itag_param url_regex -i [?&]itag=
acl sver3_param url_regex -i [?&]sver=3
cache_peer 127.0.0.1 parent 8081 0 proxy-only no-query connect-timeout=5 no-digest
cache_peer_access 127.0.0.1 allow youtube_videos id_param itag_param sver3_param !begin_param !range_request
cache_peer_access 127.0.0.1 deny all
Squid seemed to be in an infinite restarting loop and the following excerpts
from cache.log seem relevant.
The first restart had the following line in cache.log after about 2 weeks of
operation:
2012/03/25 11:23:45| assertion failed: filemap.cc:76: "fm->max_n_files<= (1<< 24)"
After that we have a rinse and repeat of squid restarting until after cache
validation and then:
2012/03/26 09:16:30| storeLateRelease: released 0 objects
2012/03/26 09:16:30| IpIntercept.cc(137) NetfilterInterception: NF getsockopt(SO_ORIGINAL_DST) failed on FD 17: (2) No such file or directory
2012/03/26 09:16:30| IpIntercept.cc(137) NetfilterInterception: NF getsockopt(SO_ORIGINAL_DST) failed on FD 19: (2) No such file or directory
[..several more of the same..]
2012/03/26 09:16:30| IpIntercept.cc(137) NetfilterInterception: NF getsockopt(SO_ORIGINAL_DST) failed on FD 132: (2) No such file or directory
2012/03/26 09:16:30| IpIntercept.cc(137) NetfilterInterception: NF getsockopt(SO_ORIGINAL_DST) failed on FD 137: (2) No such file or directory
2012/03/26 09:16:32| assertion failed: filemap.cc:76: "fm->max_n_files<= (1<< 24)"
The assertion failure is not common. A very old bugfix (Squid 2.6) suggest that
it is related to a large cache.
this line again.
I'm not sure what exactly happened. Judging from the name of the assert it had
something to do with a maximum number of files. But is it a squid limitation or
is it a filesystem limitation? Amount of filedescriptors is set to 4096.
Filesystem type is ext4.
So finally here are my questions:
1) What exactly happened and how can I fix it?
2) From your experience, are the ressources used adequate for the use case
given?
3) Is there a better way to cache video content with Squid 3.1 aside from using
a cache_peer proxy?
4) Are there other hints and tips that you could share regarding such a setup?
Thanks in advance and best regards,
- Christian Loth
