Hi all. This is currently a test environment so making changes isn't an issue. Initially I had issues with hosts updating <any flavour of Microsoft Windows> but solved that with the included squid.conf. I'm even getting real cache hits on some of the Windows XP and Windows 7 updates in my test lab, so the amount of effort I've put in so far is pretty well justified. Since the target audience won't have access to a local WSUS, I can pretty well count it as a win, even if the rest of this email becomes moot. Then came the big issue - World of Warcraft installation via the downloaded client. Things pretty well fell apart. It would install up to 20% and crash. Then it would install up to 25% and crash. Then 30% and crash. It did that, crashing further in the process each time, until it finally installed the base game (roughly 15 crashes). Due to clamping down on P2P I disabled that update mechanism and told the downloader to use only direct download. I'm averaging 0.00KB/s with bursts from 2KB/s to 64 KB/s. If I take squid out of the line I get speeds between 1 and 3 MB/s+ and things just work - but that sort of defeats the purpose in having a device that will cache non-authenticated user content. Having one user download a new 1 GB patch, and it being available locally for the other couple of hundred, would be ideal. Still, it isn't a deal breaker. I understand that it could be related to the partial content reply for the request and I understand that it could also be related to the <URL>/<foo>? style request. Is the best approach to just automatically pass anything for blizzard.com/worldofwarcraft.com straight through and not attempt to cache the updates? I've seen some comments where using acl QUERY urlpath_regex cgi-bin \? cache deny QUERY will cause those requests to not be cached (and I understand why that is) but I'm wondering if I should just ignore them altogether, especially given the third item - YouTube. The target population for this cache is rather large. Typically, youtube is a huge culprit for bandwidth usage and a lot of the times it's hundreds of people hitting the same videos. I've been looking at how to cache those and it seems like it's required to either not use the above ACL or it's to setup another ACL that specifically allows youtube. All of those comments and workarounds have been regarding the 2.x set of squid, though. I'm curious if there is a cleaner way to go about caching youtube (or, perhaps I should say, video.google.com) in 3.1.x, or if it's possible to cache things like the WoW updates now? We're looking to experiment with some proprietary devices that claim to be able to cache Windows Updates, YouTube/Google Video, etc., but I'm wondering if my woes are just because of my inexperience with squid or if they're just that far ahead in terms of functionality? Any hints, tips or suggestions would be more than welcome! Relevant version information and configuration files: fergie# squid -v Squid Cache: Version 3.1.9 configure options: '--with-default-user=squid' '--bindir=/usr/local/sbin' '--sbindir=/usr/local/sbin' '--datadir=/usr/local/etc/squid' '--libexecdir=/usr/local/libexec/squid' '--localstatedir=/var/squid' '--sysconfdir=/usr/local/etc/squid' '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid/squid.pid' '--enable-removal-policies=lru heap' '--disable-linux-netfilter' '--disable-linux-tproxy' '--disable-epoll' '--disable-translation' '--enable-auth=basic digest negotiate ntlm' '--enable-basic-auth-helpers=DB NCSA PAM MSNT SMB squid_radius_auth' '--enable-digest-auth-helpers=password' '--enable-external-acl-helpers=ip_user session unix_group wbinfo_group' '--enable-ntlm-auth-helpers=smb_lm' '--without-pthreads' '--enable-storeio=ufs diskd' '--enable-disk-io=AIO Blocking DiskDaemon' '--disable-ipv6' '--disable-snmp' '--disable-htcp' '--disable-wccp' '--enable-pf-transparent' '--disable-ecap' '--disable-loadable-modules' '--enable-kqueue' '--with-large-files' '--prefix=/usr/local' '--mandir=/usr/local/man' '--infodir=/usr/local/info/' '--build=amd64-portbld-freebsd8.1' 'build_alias=amd64-portbld-freebsd8.1' 'CC=cc' 'CFLAGS=-O2 -pipe -fno-strict-aliasing' 'LDFLAGS=' 'CPPFLAGS=' 'CXX=c++' 'CXXFLAGS=-O2 -pipe -fno-strict-aliasing' 'CPP=cpp' --with-squid=/usr/ports/www/squid31/work/squid-3.1.9 --enable-ltdl-convenience It's running in transparent mode on fergie# uname -m -r -s -v FreeBSD 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:36:49 UTC 2010 root@xxxxxxxxxxxxxxxxxxxxx:/usr/obj/usr/src/sys/GENERIC amd64 which is basically a vanilla FreeBSD 8.1 install with squid installed from ports. My squid.conf: ################################################### # # Recommended minimum configuration: # acl manager proto cache_object acl localhost src 127.0.0.1/32 ::1 acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1 # Example rule allowing access from your local networks. # Adapt to list your (internal) IP networks from where browsing # should be allowed acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT # # Recommended minimum Access Permission configuration: # # Only allow cachemgr access from localhost http_access allow manager localhost http_access deny manager # Deny requests to certain unsafe ports http_access deny !Safe_ports # Deny CONNECT to other than secure SSL ports http_access deny CONNECT !SSL_ports # We strongly recommend the following be uncommented to protect innocent # web applications running on the proxy server who think the only # one who can access services on "localhost" is a local user #http_access deny to_localhost # # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS # # Example rule allowing access from your local networks. # Adapt localnet in the ACL section to list your (internal) IP networks # from where browsing should be allowed http_access allow localnet http_access allow localhost acl windowsupdate dstdomain windowsupdate.microsoft.com acl windowsupdate dstdomain .update.microsoft.com acl windowsupdate dstdomain download.windowsupdate.com acl windowsupdate dstdomain redir.metaservices.microsoft.com acl windowsupdate dstdomain images.metaservices.microsoft.com acl windowsupdate dstdomain c.microsoft.com acl windowsupdate dstdomain www.download.windowsupdate.com acl windowsupdate dstdomain wustat.windows .com acl windowsupdate dstdomain crl.microsoft.com acl wuCONNECT dstdomain www.update.microsoft.com #no_cache deny windowsupdate #no_cache deny wuCONNECT http_access allow CONNECT wuCONNECT localnet http_access allow windowsupdate localnet # And finally deny all other access to this proxy http_access deny all # Squid normally listens to port 3128 http_port 3128 transparent # We recommend you to use at least the following line. hierarchy_stoplist cgi-bin ? # Uncomment and adjust the following to add a disk cache directory. cache_dir ufs /var/squid/cache 175000 16 256 # Cache Mem - ideal amount of RAM to use cache_mem 2048 MB # Maximum object size - default is 4MB, not nearly enough to be useful maximum_object_size 1024 MB # Maximum object size in memory - we have 4GB, we can handle larger objects maximum_object_size_in_memory 512 MB # Read Timeout - BASE can take ages to read data, even more than the default 15 minutes read_timeout 30 minutes # always pull the entire file from the start when a range is requested range_offset_limit -1 # allow full object to download when client disconnects quick_abort_min -1 KB # Leave coredumps in the first cache dir coredump_dir /var/squid/cache #refresh_pattern -i ak\.worldofwarcraft\.com\.edgesuite\.net(.*)/(.*) 1440 90% 100800 reload-into-ims override-expire #refresh_pattern -i blizzard\.vo\.llnwd\.net/(.*) 1440 90% 100800 reload-into-ims override-expire #refresh_pattern -i attdist\.blizzard\.com/(.*) 1440 90% 100800 reload-into-ims override-expire # Add any of your own refresh_pattern entries above these. refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 ################################################### My /etc/pf.conf: fergie# pfctl -f /etc/pf.conf No ALTQ support in kernel ALTQ related functions disabled fergie# cat /etc/pf.conf int_if=bge1 ext_if=bge0 rdr pass log(all) on $int_if proto tcp from any to any port 80 -> 127.0.0.1 port 3128 nat pass log(all) on $ext_if from $int_if:network to any -> ($ext_if) pass log(all) on $ext_if pass log(all) on $int_if
