On Tuesday 31 March 2009 13:32:08 Gavin McCullagh wrote: > Hi, > > running squid in transparent mode for our network, we're trying to > maximised our byte cache hit ratio. Isn't everyone ;-) > > I've been looking over reports generated by SRG and I guess shouldn't be > surprised to find that youtube/goodlevideo (when you add up the 250 > different mirror servers) accounts for around 15% of our bandwidth usage > (c. 28GB per day in c. 300 requests) and has a 0% hit rate. This seems > worthy of some effort. > > So, I've been looking at these pages: > > http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube > http://wiki.squid-cache.org/ConfigExamples/DynamicContent/YouTube/Discussio >n > > I gather I'd need to upgrade to squid 2.7 (for storeurl) which I can do. > The remainder looks doable, if a little complex. > > Is there a big performance hit involved in spawning the perl script? I > guess those ACLs will get hit relatively rarely so perhaps not? > > I guess the perl script needs to be modified every time google/youtube > change subdomain names. Do I presume this doesn't happen too often? > > Does anyone else have any experience or advice on this? What sort of hit > rate do you tend to get in practice? There's a lot of video on > goodlevideo, but I guess certain videos tend to become highly popular for a > few days. > > Many thanks in advance for any suggestions, > Gavin Hi Gavin, I've tried a lot of methods of caching youtube/googlevideos. I also tried the method you mentioned. I've decided against it, mostly because of performance issues. As squid cannot utilize SMP machines, you will be forced to use sibling relationships if you plan on exceeding 600-700~ mbit per second, and even then be prepared for a lot of performance issues. I also tried videocache (http://cachevideos.com/) which is more scaleable, and easier to maintain, but suffered from stability issues (mainly with the RPC component - it tends to hang at high loads). I ended up rewriting videocache (using perl), and I am now pushing 1.5 Gbit of youtube traffic on a single HP DL 360 G5 machine (2 Intel E5410 quad core CPUs). To summarize, I would recommend checking videocache out, as I know the developer is aware of the RPC problem and he is working to solve it.
