> > >> -----Original Message----- >> From: Amos Jeffries [mailto:squid3@xxxxxxxxxxxxx] >> Sent: 16 January 2008 21:12 >> To: Tris Hoar >> Cc: squid-users@xxxxxxxxxxxxxxx >> Subject: Re: Streaming asx files through squid >> >> > Hi List, >> > >> > We are fairly new to using squid and are in the process of migrating >> 400 >> > odd >> > proxy servers over to it. This is going fairly smoothly at the moment. >> > However some of the customers have reported that >> http://www.heartfm.co.uk >> > is >> > no longer streaming radio for them. >> > >> > The server that the audio streams from firstly sends this HTML file to >> the >> > client >> > >> > <asx version = "3.0"> >> > <title>Heart West Midlands 100.7 Live Stream</title> >> > <entry> >> > >> > <title>LIVE : >> > Heart1007</title> >> > >> > <ref >> > href="http://wm-global.london.as34763.net/Heart1007"; /> >> > >> > </entry> >> > >> > </asx> >> > >> > >> > Then http://wm-global.london.as34763.net/Heart1007 >> > >> > >> > >> > Points to >> > >> > >> > >> > [Reference] >> > >> > Ref1=http://wm-global.london.as34763.net/Heart1007?MSWMExt=.asf >> > >> > Ref2=http://192.168.97.221:80/Heart1007?MSWMExt=.asf >> > >> > >> > >> > If we use http://wm-global.london.as34763.net/Heart1007?MSWMExt=.asf >> > directly in Media player 11 this works through squid >> > >> > >> > >> > Our setup is a local squid server at each site > to DNS round robin of >> > parent servers in the core > Internet >> > >> > The squid servers are built on Xeon 3.0 GHz servers with 2GB ram and >> > ~500GB >> > disk space of which 400GB is cache. The OS is REHL 5.1 and running >> Squid >> > 2.6.STABLE6 (we are using this version due to redhat offering support >> for >> > it) >> >> For that size disk cache the RAM is very low. A rule of thumb for >> general >> principles is to dedicate 10MB-RAM per 1 GB-disk just to hold the cache >> index. If your primary business is streaming media you can get away with >> less (very-large files need less index per disk byte). > > The customers are Schools and many of them have subscriptions to services > like Espresso and Knowledgebox. These services are web based and have a > lot > of video content on them. The plan is to pre populate most of the cache > space with the video content using an application called cachestream. Thus > the low memory should not be a problem. > >> >> > >> > Using the parents directly or the old proxy server's this streams fine >> > though windows media player 11. >> > >> > >> > >> > Our squid config is >> > >> > ------------------------------------------------------------------------ >> ---- >> > >> > http_port 80 >> >> Is this meant to be a website accelerator or customer proxy? >> >> Accelerators need accel or vhost and could do with defaultsite= in their >> http_port. >> > > The servers are intended to function as a standard web proxy's, not as > accelerators. In which case port 80 is not the best choice to open. Particularly in light of the unrestrictive ACLs below. > >> >> > >> > hierarchy_stoplist cgi-bin ? >> > >> > acl QUERY urlpath_regex cgi-bin \? >> > >> > cache deny QUERY >> > >> >> Are you sure you don't want to cache anything with '?' in the URI? >> The dev team are now recommending killing this and using a >> refresh_pattern >> instead to allow caching of properly working dynamic pages/files. >> > > We were just using the example included with the default squid.conf for > this. Do you have an example of a refresh_pattern we could try? We are now recomending a default set of refresh_patterns: refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern \? 0 0% 0 refresh_pattern cgi-bin 0 0% 0 refresh_pattern . 0 20% 4320 Instead of these: acl QUERY urlpath_regex cgi-bin \? cache deny QUERY > >> >> > >> > acl apache rep_header Server ^Apache >> > >> > broken_vary_encoding allow apache >> > >> > >> > cache_replacement_policy heap LRU >> > >> > memory_replacement_policy heap GDSF >> > >> > cache_dir aufs /var/spool/squid 400000 64 256 >> > >> > access_log /var/log/squid/access.log >> > >> > access_log none >> > >> >> This second access_log line is not correct. 'none' is not a proper >> filename for the log, did you mean 'cache_log none'?. >> I would recommend having a cache_log with 'debug_options ALL,0' so that >> you get a log of even just the critical errors squid has. >> > > Opps! We have a duplicated line > access_log /var/log/squid/access.log <- should not have been there > We monitor this from our core servers rather than the edge devices. Though > while testing I will have the log on. > >> > >> > refresh_pattern ^ftp: 1440 20% 10080 >> > >> > refresh_pattern ^gopher: 1440 0% 1440 >> > >> > refresh_pattern . 0 20% 4320 >> > >> > >> > >> > >> > >> > acl all src 0.0.0.0/0.0.0.0 >> > >> > acl manager proto cache_object >> > >> > acl localhost src 127.0.0.1/255.255.255.255 >> > >> > acl to_localhost dst 127.0.0.0/8 >> > >> > acl SSL_ports port 443 563 8443 >> > >> > acl Safe_ports port 80 81 21 443 563 70 210 280 554 488 591 777 889 >> > 1025-65535 >> > >> > acl CONNECT method CONNECT >> > >> > acl Proxy_bypass dstdomain .misportal.net .bgfl.org >> > >> > >> > >> > >> > >> > #Access Control Rules >> > >> > http_access allow manager localhost >> > >> > http_access deny manager >> > >> > http_access deny !Safe_ports >> > >> > http_access deny CONNECT !SSL_ports >> > >> > http_access allow localhost >> > >> > http_access allow all >> >> Oh yay! An open web proxy on a high-speed server. >> >> If this is actually a site accelerator note the comment after http_port >> above, and you will need some ACL restricting the destination sites by >> name or rDNS. Probably listing some cache_peer's as preferred >> information >> sources. >> > > The proxy sits out at the customer's site which connects back to our > servers > over a 10MB MPLS connection. This connection is firewalled from the > internet > and restricted with ACL's on the routers at each site so that only people > at > that site or at our office can connect to the server. Every one at the > site > MUST use the proxy for internet access however we do not wish to use a > transparent proxy. Rather than having a custom config for each site we > allow > any one to connect to the server as only people at that site will be able > to. > >> > >> > http_reply_access allow all >> > >> > icp_access allow all >> > >> > >> > >> > always_direct allow Proxy_bypass >> > >> > never_direct allow all >> > >> > >> > >> > nonhierarchical_direct off >> > >> > >> > >> > coredump_dir /var/spool/squid >> > >> > visible_hostname eduproxy.bgfl.org >> > >> > >> > >> > cache_effective_user squid >> > >> > cache_effective_group squid >> > >> > cachemgr_passwd xxxxxx info stats/objects >> > >> > >> > >> > ------------------------------------------------------------------------ >> ---- >> > >> > >> > >> > If you can shed any light on how to resolve this that would be great! >> > >> >> Firstly check that the clients software is even using the proxy. >> This should show up as entries in access.log for the stream URI. >> > > This is showing up in the access log fine. > TCP_MISS/200 6310 GET http://meta.as34763.net/content/9.asx - > FIRST_UP_PARENT/parentproxy.bgfl.org video/x-ms-asf > >> Then enable a cache_log and see if there are any serious problems >> happening in squid about that time. > > Nothing showing up in the cache_log that seams wrong > >> >> Then check the request headers going to/from squid for one of the >> failing >> requests. Also, does it work with any other software? and what does that >> show? >> > > Looking at TCPDump I think the problem is due to the streaming server > (which > we do not control) trying to talk in HTTP1.1 and Squid not understanding > it. > Unless we can find a better solution what we are planning to do is set the > core servers to just rewrite the url from > http://meta.as34763.net/content/9.asx > Into > http://wm-global.london.as34763.net/Heart1007?MSWMExt=.asf > > >> If that does not lead you to a solution, we will need to know the squid >> release (including the stable part) and what you have managed to find >> out >> from trying the above. >> >> > > 2.6.Stable6 - this is the RedHat maintained release. Aha, older release + broken streaming server. Try it with a later squid on general principles. We have been seeing more and more of these problems with broken HTTP/1.1 servers and are adding compatibility hacks with nearly every release now. Amos
