On 22/02/11 16:41, Eliezer wrote:
Thank amos and everyone that helped me.
today i finished my youtube caching project using squid 3.1.11 and squid2.7
it took me awhile to understand how everything works and all the weird
stuff that i got while rewriting the address and other stuff.
André Luiz dos Santos helped me very very much seeing some things that i
haven't seen about some end-points\states on youtube and other video sites.
i had a lot of trouble cause the urls for the videos are hell and i dont
like regular expressions.
i had an exam on java programming so to prepare for the test i played
with java and made the store_url_rewrite program on java.
the nice thing about this software is the simplicity of that i could
make things understandable and make the debugging steps for problems
much more effective.
André gave me the nice link to his work on:
http://code.google.com/p/youtube-cache/
i prefer the store_url_rewrite cause it's a cache server and not video
storage one so nginx and other stuff are nice but not exactly what i and
many other want to achieve.
problems i encounter on the way:
- identifying youtube dynamic urls (there are many of the) patterns.
- return =NULL logged as return from the storeurl_program
the culprit was too much storeurl_rewrite_children
using java storeurl_rewrite_children 2 is more then anyone can need and
more then 15 causing the problem.
- i got weird things like "rewritten url part one" + IP/- - GET -
myip=IP myport=3128"second end of the url"
until now i really dont know the cause for it and the only option is the
my store_rewrite program was analysing wrong the input url
and for that i have changed my code to make sure that the output is fixed.
Looks like debugging trace text being sent to stdout instead of stderr.
The NULL result may be related to this problem. Squid routinely handles
many hundreds of simultaneous helpers for some networks.
- sometimes the youtube links are not the videos them-self but a
redirection link to other domains and to avoid gettings "error loading"
i needed to change the minimum object size to cache.
- the best way i have found till now to make the cache reliable is to
actually KNOW what the refresh rules are and all the acls system works.
if you will add the correct rules you will benefit a lot.
:)
- in the squid-cache site\wiki some of the explanations are old\not
understable.
- i didnt look that much but i am looking for a documented VISUAL IMAGE
of the squid system such as in iptables in here:
http://www.linuxhomenetworking.com/wiki/images/f/f0/Iptables.gif
i would be glad to make one if there isn't.
- i am looking for a way to extend an object cache lease\life span based
on usage.
is there a way to "mangle" a single object cache validity without
releasing and reloading it into the cache?
The HTTP validation requests do this, extending the cached age
dynamically based on the difference between modification time and
validation time. refresh_pattern is how you tune the rate of
re-validation. To work nicely this requires the origin server to answer
validation checks with 304 responses as appropriate instead of always
re-sending a whole new object.
Amos
--
Please be using
Current Stable Squid 2.7.STABLE9 or 3.1.11
Beta testers wanted for 3.2.0.5
