Hi, I've been trying to set up squid for a variety of reasons, but one is to cache commercials from HULU. HULU takes up a huge portion of my bandwidth. It'd be pointless to try to cache the TV shows since we typically only watch them once (I think they're encrypted anyway), but I'd like to cache the unencrypted commercials since those get played over and over again. The commercials come in as partial content (206 code), so I wrote a little script that grabs the list of videos with a 206 code each night and does a "wget -O /dev/null [full address for video]" to cache the commercials for the next day. This seems to be working just fine. The problem is, it still will not cache the videos (even though it is no longer a partial content request). Can someone help sort me out? (P.S. I'm very much a beginner, so my config file is a result of just copying a lot of stuff off the internet... I'll gladly take suggestions even if its not directly related to the HULU ads.) Thanks in advance for your help! store.log: 1404923465.907 RELEASE -1 FFFFFFFF B493FE4264AFC396D382596A22280AA5 200 1404923474 1403637739 1436459474 video/mp4 12018601/12018601 GET http://ads-v-darwin.hulu.com/published/2014/6/24/7/826d2864_2b05_4dd7_9ced_c9992a1dd04d_12d382_AdsTranscode_16x9_1403420_24623827_H264_3200.mp4 access.log: 1404923465.907 4383 192.168.1.80 TCP_MISS/200 12019050 GET http://ads-v-darwin.hulu.com/published/2014/6/24/7/826d2864_2b05_4dd7_9ced_c9992a1dd04d_12d382_AdsTranscode_16x9_1403420_24623827_H264_3200.mp4 - HIER_DIRECT/8.27.246.254 video/mp4 # ACCESS CONTROLS # ----------------------------------------------------------------------------- ## local network #acl localnet src 10.0.0.0/8 # RFC1918 possible internal network #acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines ## windows update acl windowsupdate dstdomain windowsupdate.microsoft.com acl windowsupdate dstdomain .update.microsoft.com acl windowsupdate dstdomain download.windowsupdate.com acl windowsupdate dstdomain redir.metaservices.microsoft.com acl windowsupdate dstdomain images.metaservices.microsoft.com acl windowsupdate dstdomain c.microsoft.com acl windowsupdate dstdomain www.download.windowsupdate.com acl windowsupdate dstdomain wustat.windows.com acl windowsupdate dstdomain crl.microsoft.com acl windowsupdate dstdomain sls.microsoft.com acl windowsupdate dstdomain productactivation.one.microsoft.com acl windowsupdate dstdomain ntservicepack.microsoft.com ## hulu videos acl huluvids dstdomain ads-v-darwin.hulu.com acl huluvids dstdomain assets.hulu.com acl huluvids dstdomain assets.huluim.com acl huluvids dstdomain p.hulu.com acl huluvids dstdomain m.hulu.com acl huluvids dstdomain ib.huluim.com acl huluvids dstdomain s.hulu.com ## do not cache sites acl directsites dstdomain "/etc/squid3/go_direct.txt" ## malware sites acl malwareblockedsites dstdomain "/etc/squid3/malware_blocked_sites" ## fix for netflix on tv acl VIERAWMDRM url_regex ^http://us.vieraconnect.tv/wmdrm$ ## basic ports acl SSL_ports port 443 acl SSL_ports port 563 acl SSL_ports port 81 acl SSL_ports port 2087 acl SSL_ports port 10000 acl Safe_ports port 563 acl Safe_ports port 631 acl Safe_ports port 901 acl Safe_ports port 81 acl Safe_ports port 3128 acl Safe_ports port 1025-65535 acl Safe_ports port 210 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl purge method PURGE acl CONNECT method CONNECT ## windows update acl wuCONNECT dstdomain www.update.microsoft.com acl wuCONNECT dstdomain sls.microsoft.com http_access allow CONNECT wuCONNECT localnet http_access allow windowsupdate localnet http_access allow huluvids http_access deny malwareblockedsites http_access allow localnet cache allow huluvids cache deny VIERAWMDRM cache deny directsites http_access deny !Safe_ports http_access deny CONNECT !SSL_ports # Only allow cachemgr access from localhost http_access allow localhost manager http_access deny manager http_access allow localhost http_access deny all # LISTENING PORTS # ----------------------------------------------------------------------------- http_port 3128 http_port 192.168.1.3:3129 transparent # MEMORY CACHE OPTIONS # ----------------------------------------------------------------------------- cache_mem 256 MB maximum_object_size_in_memory 512 KB memory_replacement_policy heap GDSF # DISK CACHE OPTIONS # ----------------------------------------------------------------------------- cache_replacement_policy heap LFUDA cache_dir aufs /var/cache/squid 50176 14 256 maximum_object_size 2048 MB cache_swap_low 96 cache_swap_high 98 # LOGFILE OPTIONS # ----------------------------------------------------------------------------- access_log daemon:/var/log/squid3/access.log squid cache_store_log daemon:/var/log/squid3/store.log logfile_rotate 2 # OPTIONS FOR TROUBLESHOOTING # ----------------------------------------------------------------------------- cache_log /var/log/squid3/cache.log coredump_dir /var/spool/squid3 # OPTIONS FOR TUNING THE CACHE # ----------------------------------------------------------------------------- refresh_pattern -i (p|m|s).hulu.com/.*\.(mp?) 4320 80% 43200 reload-into-ims override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i ib.huluim.com/.*\.(mp?) 4320 80% 43200 reload-into-ims override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i assets.hulu*.com/.*\.(mp?) 4320 80% 43200 reload-into-ims override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i .*ads-v-darwin.hulu.com.* 4320 80% 43200 reload-into-ims override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims refresh_pattern -i windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims refresh_pattern -i windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip) 4320 80% 43200 reload-into-ims refresh_pattern -i (get_video\?|videoplayback\?|videodownload\?) 4320 80% 43200 override-expire ignore-reload reload-into-ims ignore-no-cache ignore-private refresh_pattern -i ([^.]+.|)symantecliveupdate.com/.*\.(zip|exe) 4320 100% 43200 reload-into-ims refresh_pattern -i http://*.googlesyndication.*/.* 720 100% 4320 refresh_pattern -i imdb\.com 10080 90% 43200 ignore-reload override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i thetvdb\.com 10080 90% 43200 ignore-reload override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i themoviedb\.org 10080 90% 43200 ignore-reload override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i impawards\.com 10080 90% 43200 ignore-reload override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern -i movieposterdb\.com 10080 90% 43200 ignore-reload override-expire ignore-no-cache ignore-no-store ignore-private refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i \.(gif|png|jp?g|ico|bmp|tiff?)$ 10080 95% 43200 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-private refresh_pattern -i \.(rpm|cab|deb|exe|msi|msu|zip|tar|xz|bz|bz2|lzma|gz|tgz|rar|bin|7z|doc?|xls?|ppt?|pdf|nth|psd|sis)$ 10080 90% 43200 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-private refresh_pattern -i \.(avi|iso|wav|mid|mp3|mp4|mpeg|mov|3gp|wm?|swf|flv|x-flv|axd)$ 43200 95% 86400 override-expire override-lastmod reload-into-ims ignore-no-cache ignore-private refresh_pattern -i \.(html|htm|css|js)$ 1440 75% 40320 refresh_pattern -i \.index.(html|htm)$ 0 75% 10080 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern (Release|Packages(.gz)*)$ 0 20% 2880 refresh_pattern . 0 40% 40320 ## always download the whole file even if client aborts quick_abort_min -1 ## TAG: range_offset_limit size [acl acl...] range_offset_limit 2048 MB windowsupdate range_offset_limit 2048 MB huluvids ## Used for estimating number of objects store_avg_object_size 30 KB # HTTP OPTIONS # ----------------------------------------------------------------------------- # TAG: request_header_access request_header_access From deny all request_header_access Server deny all request_header_access Link deny all request_header_access Via deny all request_header_access X-Forwarded-For deny all # TIMEOUTS # ----------------------------------------------------------------------------- forward_timeout 240 seconds connect_timeout 30 seconds peer_connect_timeout 5 seconds read_timeout 600 seconds request_timeout 60 seconds shutdown_lifetime 60 seconds # ADMINISTRATIVE PARAMETERS # ----------------------------------------------------------------------------- cache_effective_user proxy cache_effective_group proxy # TAG: httpd_suppress_version_string on|off # Suppress Squid version string info in HTTP headers and HTML error pages. #Default: httpd_suppress_version_string on # ICP OPTIONS # ----------------------------------------------------------------------------- log_icp_queries off # MISCELLANEOUS # ----------------------------------------------------------------------------- memory_pools on client_db on reload_into_ims on
