Hi Amos,
If some of these entries are incorrect other than the redirect rule,
it's unfortunately a bug with the pfSense module. pfSense uses a nice
little web GUI to create the squid.conf. I don't have the ability to
edit anything you see below except the rules I've defined are created
automatically by pfsense. If I overwrite it, I'll lose my changes when
it restarts. I'll take a look at the code as it's written in PHP and
submit a feature request with a patch to clean up a lot of the
boilerplate configuration. Thanks for all the help, you've been a great
resource to get this working quickly.
Todd
Amos Jeffries wrote:
On Tue, 29 Sep 2009 13:22:58 +1300, Todd Nine <todd@xxxxxxxxxxxxxxxxxx>
wrote:
Hi Amos,
Here is my squid.conf. I've just used the defaults and added a single
rule. We're pushing a lot of throughput (several gigs a day). I've
disabled writing to disk as we actually run from a USB appliance, and
set the cache size to 1 GB (1024M) of RAM. My main use of squid is not
caching, but rather http redirection to save us money on our usage fees
from our ISPs.
In which case you probably do want to look at caching. Since that can have
between 20% and 40% reduction in HTTP traffic going to your ISPs.
What version of squid is this?
Thanks again for the help!
File:
# Do not edit manually !
http_port 10.0.1.1:3128
http_port 10.0.2.1:3128
http_port 127.0.0.1:80 transparent
icp_port 0
pid_filename /var/run/squid.pid
cache_effective_user proxy
cache_effective_group proxy
error_directory /usr/local/etc/squid/errors/English
icon_directory /usr/local/etc/squid/icons
visible_hostname router
cache_mgr admin@localhost
access_log /var/squid/log/access.log
cache_log /var/squid/log/cache.log
cache_store_log none
shutdown_lifetime 3 seconds
# Allow local network(s) on interface(s)
acl localnet src 10.0.1.0/255.255.255.0 10.0.2.0/255.255.255.0
Please use CIDR masks:
acl localnet src 10.0.1.0/24 10.0.2.0/24
uri_whitespace strip
cache_dir aufs /var/squid/cache 100 16 256
Huh? you said you disabled writing to disk. That above is using a 100MB
cache on the disk.
To disable disk caching use the 'cache_dir null ' storage type in Squid
older than 3.1, or remove all cache_dir from Squid-3.1+.
cache_mem 1024 MB
maximum_object_size 4 KB
1GB worth of 4KB objects is a LOT of objects. If you have the memory to
spare 1GB for caching its probably best to allow moderate sized objects to
be cached in RAM. Setting Max size to 1MB should do. Though depending on
the popularity of video sites with your users they may also benefit from
10MB max object size (video causes a bump at 2MB-8MB apparently).
minimum_object_size 0 KB
cache_replacement_policy heap LFUDA
memory_replacement_policy heap GDSF
offline_mode off
dns_children 32
The above is only relevant with the obsolete 'dnsserver' helper. If you are
still using that you would get a great deal of performance boots by
changing to the internal DNS (requires a recompile).
cache_swap_low 90
cache_swap_high 95
acl donotcache dstdomain "/var/squid/acl/donotcache.acl"
cache deny donotcache
# No redirector configured
# Setup some default acls
acl all src 0.0.0.0/0.0.0.0
acl localhost src 127.0.0.1/255.255.255.255
Some more CIDR benefits:
acl all src all
acl localhost src 127.0.0.1
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 1111
3128 1025-65535
acl sslports port 443 563 1111
acl manager proto cache_object
acl purge method PURGE
acl connect method CONNECT
acl dynamic urlpath_regex cgi-bin \?
cache deny dynamic
More speed boost by dropping the above QUERY
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !safeports
http_access deny CONNECT !sslports
# Always allow localhost connections
http_access allow localhost
request_body_max_size 0 KB
reply_body_max_size 0 allow all
delay_pools 1
delay_class 1 2
delay_parameters 1 -1/-1 -1/-1
delay_initial_bucket_level 100
delay_access 1 allow all
# Allow local network(s) on interface(s)
http_access allow localnet
# Custom options
#Set up our ACL for high throughput sites
acl high_throughput dstdomain .amazonaws.com .rapidshare.com
#Bind high throughput to the wireless interface
tcp_outgoing_address 116.90.140.xx high_throughput
# Default block all to be sure
http_access deny all
Amos Jeffries wrote:
On Tue, 29 Sep 2009 09:32:49 +1300, Todd Nine <todd@xxxxxxxxxxxxxxxxxx>
wrote:
Thanks for the help! I read over the rules and it was quite easy to
set
up what I needed once I had the right directive. I simply set up the
following.
#Set up our ACL for high throughput sites
acl high_throughput dstdomain .amazonaws.com
#Bind high throughput to the wireless interface
tcp_outgoing_address 116.90.140.xx high_throughput
However we're having a side effect issue. Our router box is a bit old
(an old P4), and we can't keep up with the squid demands due to the
number of users with 2 GB of ram. Is there a directive that I can tell
squid not to proxy connections unless they meet the "high_throughput"
acl? I looked and couldn't find any bypass directives that met what I
needed.
Thanks,
Todd
Once connections have already entered Squid its too late to not send
them
to Squid.
I have run Squid on P4s routers with 256MB of RAM for hundreds of
domains
and dozens of clients without having the box run up much of a sweat.
What
is your load (both CPU box load, and visitor rates, bandwidth) like?
Also check your other configuration and access controls are using
efficient
methods, if you don't know what those are already I'm happy to give
configs
an audit and point things that need adjusting out.
Amos
Amos Jeffries wrote:
On Mon, 28 Sep 2009 16:21:16 +1300, Todd Nine
<todd@xxxxxxxxxxxxxxxxxx>
wrote:
Hi all,
I'm using squid on a pfSense router we've built. We have 2
connections, one we pay for usage (DSL) and one we do not (Wireless).
We use Amazon S3 extensively at work. We've been attempting to route
all traffic over the wireless via an IP range, but as S3 can change
IPs,
this doesn't work and we end up with a large bill for our DSL. Is it
possible to have squid route connections via a specific interface if
a
hostname such as "amazonaws.com" is in the HTTP request header?
Thanks,
Todd
Yes you can.
Find an IP assigned to the interface you want traffic to go out. Use
the
tcp_outgoing_addr directive and ACLs that match the requests to make
sure
all the requests to that domain are assigned that outgoing address.
Then
make sure the OS sends traffic from that IP out the right interface.
Amos
--
todd
SENIOR SOFTWARE ENGINEER
todd nine | spidertracks ltd | 117a the square
po box 5203 | palmerston north 4441 | new zealand
P: +64 6 353 3395 | M: +64 210 255 8576
E: todd@xxxxxxxxxxxxxxxxxx <mailto:todd@xxxxxxxxxxxxxxxxxx>
W: www.spidertracks.com <http://www.spidertracks.com>
