Search squid archive

RE: Updated CentOS/Squid/Tproxy Transparency steps.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Amos did this already....although the wiki article needs some
corrections because Amos merged the older with the newer. I need to get
that information to him. The steps, if followed in the wiki article may
not work quite right.

Nick


-----Original Message-----
From: Henrik Nordstrom [mailto:henrik@xxxxxxxxxxxxxxxxxxx] 
Sent: Monday, June 29, 2009 4:01 PM
To: Ritter, Nicholas
Cc: squid-users
Subject: Re:  Updated CentOS/Squid/Tproxy Transparency
steps.

Would be great if you could dump this in a wiki article to make it
easier to find (and update if needed).


Regards
Henrik

tor 2009-06-25 klockan 12:30 -0500 skrev Ritter, Nicholas:
> Some assumptions:
> 
> 1) You are using a Cisco Router to redirect traffic to the squid box
via
> WCCP
> 
> 2) 12.4(15)T8 or higher IOS on the router
> 
> 3) In my setups, the squid box is always Layer 2 adjacent to the Cisco
> router, either through a dedicated interface, or a sub-interface.
> 
> 4) The ability to compile and install a Linux kernel. Please note that
> in these steps, I am NOT using a redhat kernel, nor am I using the
> RedHat method of building a kernel.
> 
> 5) Some steps outlined here can be achieved through several different
> means, follow the steps exactly before emailing me or the list, as I
> have tested other methods, and they don't always work (case in point:
> GRE tunnel interface creation.)
> 
> 6) This setup assumes a separate WCCP service group for each direction
> of the HTTP connection, this is not needed, but makes the setup more
> scalable. If you choose to do it a different way, then YMMV. 
> 
> In the kernel build specific steps, I actually include possibly to
much
> information, as well as tell you to enable things that are not always
> needed for TPROXY related functionality, or never related to TPROXY
> functionality. I included them because they fit more environments, and
> thus less time wasted by people asking me questions, not that I mind
but
> I don't have enough time to answer all the emails I get. I tried to
> prepare this information out without errors, if the steps don't work,
> email me with the details of where you had problems so that I can
adjust
> the steps below. At the end the steps below are some common things to
> watch for in the steps that can cause the setup not to work.
> 
> 
> Steps
> 
> 1) Install CentOS 5.3, make sure you install nothing other than the
base
> packages, and trim even those down. I tend to install specific
packages
> from the distro later. 
> 
> Note: I suggest that you make separate partition(s) for where squid
will
> actually store its caches. Later mount these partitions with specific
> options (like "noatime") that will help increase performance.
> 
> 2) In the initial ncurses-based setup screen, turn off services that
you
> don't need, and turn off selinux compeletely.
> 
> 3) After install and initial bootup and configuration, run "yum
update"
> to update the system for fixes, etc. Then reboot.
> 
> 4) After step 2, issue this yum command:
> 
> yum install libcap libcap-devel gcc gcc-c++ bison flex yacc autoconf
> automake ncurses ncurses-devel rpm-devel libpcap tcpdump
> 
> Note: let it install other dependency packages. The command above
> installs compiles, utilities, etc.
> 
> 
> 5) Download iptables-1.4.3.2 from netfilter.org
> 
> 6) Download kernel 2.6.30 from kernel.org
> 
> 7) Download squid-3.1.0.8 from squid.org
> 
> 8) Decompress the kernel source, I decompress it to /usr/src/,
although
> I have read all over the place that this is a bad thing to do. The
> location really does not have to be /usr/src/
> 
> 9) Go into the kernel source directory, issue the following command:
> cp /boot/config-2.6.18-128.1.14.el5 ./RH-config-boxed.config
> 
> 10) Issue this command: make menuconfig
> 
> 11) When the ncurses-based kernel config screen loads, select the
"Load
> an Alternate Configuration File" and type in the full path to the
> RH-config-boxed.config. This will load the current kernel config, and
> there may be some errors, all of which can be ignored.
> 
> 
> 12) Configure the kernel as you normally would, but be sure to enable
> the following:
> 
> In "Networking support -> Networking options"
> 
> Enable (not as modules):
> Packet socket
> Packet socket: mmapped IO
> TCP/IP networking
> IP: advanced router
> IP: policy routing
> 
> Enable (as modules):
> IP: tunneling
> 
> Enable (not as modules):
> IP: GRE tunnels over IP
> IP: broadcast GRE over IP
> Network packet filtering framework (Netfilter)
> 
> 
> In "Networking support -> Networking options -> Network packet
filtering
> framework (Netfilter)"
> 
> Enable (not as modules):
> Advanced netfilter configuration
> 
> 
> In "Networking support -> Networking options -> Network packet
filtering
> framework (Netfilter) -> Core Netfilter 
> 
> Configuration"
> 
> Enable (as modules):
> Netfilter connection tracking support
> 
> Enable (not as modules):
> Connection tracking security mark support
> Connection tracking events
> 
> Enable (as modules):
> Connection tracking netlink interface
> Transparent proxying support (EXPERIMENTAL)
> Netfilter Xtables support (required for ip_tables)
> "CONNMARK" target support
> "MARK" target support 
> "TPROXY" target support (EXPERIMENTAL)
> "connmark" connection mark match support
> "conntrack" connection tracking match support
> "mark" match support
> "socket" match support (EXPERIMENTAL)
> "state" match support
> 
> 
> In "Networking support -> Networking options -> Network packet
filtering
> framework (Netfilter) -> IP: Netfilter Configuration"
> 
> Enable (as modules):
> IPv4 connection tracking support (required for NAT)
> IP tables support (required for filtering/masq/NAT)
> Full NAT
> MASQUERADE target support
> REDIRECT target support
> Packet mangling
> 
> 
> 13) After setting the above options, and any other items you want,
exit
> out of the kernel config, saving your changes. It will save the kernel
> compile config to RH-config-boxed.config so issue the following
command
> to put the new config in the right 
> 
> place:
> 
> cp RH-config-boxed.config config-centos
> 
> Then do the make, make_install_modules, make install, if no errors,
> adjust grub.conf to boot to the new kernel. The reboot to 
> 
> the new kernel.
> 
> 14) Assuming the kernel compiled, installed and booted properly, it is
> time to update iptables. Decompress the iptables 
> 
> source, and use the following configure command:
> 
> ./configure --enable-devel --enable-libipq --bindir=/bin
--sbindir=/sbin
> --sysconfdir=/etc --with-kernel=<path to new kernel source dir>
> --with-kbuild=<path to new kernel source dir> --with-ksource=<path to
> new kernel source dir>
> 
> 
> Then do the make, make install
> 
> 
> 
> 15) Edit ld.so.conf to add a library path:
> 
> vim /etc/ld.so.conf
> 
> add a line to the end of the file: /usr/local/lib
> 
> 
> 16) I suggest rebooting, just to make sure that iptables upgrade is
> working. Do a reboot, then do a "service iptables status" to make sure
> that iptables is running fine. If it is not running ok, it will show
> either a status failed, or an empty rule set.
> 
> 17) Assuming that the iptables upgrade is working fine, the next steps
> are to add the rules and interfaces needed for WCCP and TPROXY
> functionality. First, issue a "insmod ip_gre", then do a "dmesg |
tail"
> and make sure that you see "GRE over IPv4 tunneling driver" if you
don't
> see anything, or you get an error from insmod, you either compiled GRE
> not as a kernel module, or not at all.
> 
> 18) Create the GRE tunnel interface, issue "ifconfig gre0 <ip address
of
> squid server> netmask <netmask of squid server ip> up"
> 
> 19) Issue an "ifconfig" and make sure you see a gre0 interface.
> 
> 20) Next we need to add iptables rules to allow traffic to the gre
> interface itself, gre protocol traffic across the Ethernet interface,
> and WCCP traffic from the router. We will do this by editing the
> iptables save file directly. Do the following:
> 
> a.) service iptables save
> b.) vim /etc/sysconfig/iptables
> c.) just after the line that says ":RH-Firewall-1-INPUT - [0:0]" add
the
> following:
> 	-A INPUT -i gre0 -j ACCEPT 
> 	-A INPUT -p gre -j ACCEPT 
> 	-A INPUT -i eth0 -p gre -j ACCEPT
> d.) Somewhere lower in the file where the "-A RH-Firewall-1-INPUT"
rules
> are, add the following:
> 	-A RH-Firewall-1-INPUT -s <address of cisco router>/32 -p udp -m
> udp --dport 2048 -j ACCEPT 
> 	
> 	NOTE: The "<address of cisco router>" in this instance is the ip
> address of the router that the squid box itself uses. 
> 
> Port 2048 is the port that WCCP traffic uses.
> 
> e.) save the file and exit vim. Do a "service iptables restart" to
make
> sure you don't get any errors.
> 
> 
> 21) Next we add the TPROXY related iptables rules, from the command
line
> prompt issue the following commands:
> 
> iptables -t mangle -N DIVERT
> iptables -t mangle -A DIVERT -j MARK --set-mark 1
> iptables -t mangle -A DIVERT -j ACCEPT
> iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
> iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY
> --tproxy-mark 0x1/0x1 --on-port <port squid listens to> --on-ip 
> 
> <squid server ip>
> 
> 22) Assuming the commands in step 21 didn't give errors, do a "service
> iptables save"
> 
> 23) We need to edit the iptables rule order, to make sure it is in the
> correct order: "vim /etc/sysconfig/iptables" and make sure the section
> at the bottom of the file, from the ":DIVERT - [0:0]" onward looks
> something like:
> 
> -A DIVERT -j MARK --set-xmark 0x1/0xffffffff
> -A DIVERT -j ACCEPT
> -A PREROUTING -p tcp -m socket -j DIVERT 
> -A PREROUTING -p tcp -m tcp --dport 80 -j TPROXY --on-port <squid
port>
> --on-ip <squid server ip> --tproxy-mark 0x1/0x1 
> 
> When doing the cli commands in step 21, iptables puts the rules in the
> wrong order for them to work for TPROXY.
> 
> Save the file and exit vim.
> 
> 
> 24)  do a "service iptables restart"
> 
> 26) Issue the following commands, per the TPROXY wiki article: 
> 
> ip rule add fwmark 1 lookup 100
> ip route add local 0.0.0.0/0 dev lo table 100
> 
> If no errors from the two commands, add them to the end of
> /etc/rc.d/rc.local
> 
> 
> 27) I'm not sure this is needed anymore, but it doesn't seem to break
> anything, so add the following line to the end of /etc/rc.d/rc.local:
> echo 1 > /proc/sys/net/ipv4/ip_nonlocal_bind
> 
> 28) In the beginning of the rc.local file, add the following two
lines:
> 
> modprobe ip_gre
> <the ifconfig gre0 command line from step 18 above>
> 
> 
> 29) Edit /etc/sysctl.conf, make sure there is a line somewhere that
> says:
> 
> # Controls IP packet forwarding
> net.ipv4.ip_forward = 1
> 
> 30) Reboot the server
> 
> 31) Next, build you squid 3.1.x.x, make sure that pass
> "--enable-linux-netfilter" to the configure directive, build and
install
> squid.
> 
> 32) You will need to edit the squid config to make sure that it has
what
> it needs to use TPROXY and WCCP. For tproxy, you need one item, the
> "http_port" directive needs tproxy appended to it, I do the following:
> http_port <ip of squid server>:<port to bind to, "squid port" from
step
> 23 above> tproxy disable-pmtu-discovery=always
> 
> 33) Configure the WCCP specific related items in the squid config
file,
> specifically:
> 
> 	a) wccp2_router <router address used in the steps above>
> 	b) wccp_version 2
> 	c) wccp2_rebuild_wait on
> 	d) wccp2_fowarding_method 1
> 	e) wccp2_return_method 1
> 	f) wccp2_assigment_method 1
> 	g) wccp2_service dynamic 80
> 	h) wccp2_service dynamic 90
> 	i) wccp2_service_info 80 protocol=tcp flags=src_ip_hash
> priority=240 ports=80
> 	j) wccp2_service_info 90 protocol=tcp
> flags=dst_ip_hash,ports_source priority=240 ports=80
> 
> 34) There are other squid configuration items that are need which are
> not included here because they are not specific to WCCP/TPROXY
> functionality and the specifics vary by environment.
> 
> 35) In the router configuration, you need to do at least the
following:
> 	a.) enable wccp globally with (this might actually be optional):
> ip wccp web-cache
> 	b.) enable the specific services:
> 		1.) ip wccp 80
> 		2.) ip wccp 90
> 		Note: I use a redirect list ACL with the two commands
> above so that the router doesn't WCCP redirect specific web sites. The
> command would look like "ip wccp 80 redirect-list 122", and
access-list
> 122 would be a list of denies for sites to not be redirected, and a
> permit any any at the end of the access-list to allow all other
websites
> to be wccp redirected.
> 	c.) On the interface that is adjacent to the squid box, do a "ip
> wccp redirect exclude in" this command makes it so that the router
does
> not redirect the squid traffic as well as other client traffic.
> 
> 
> 
> Some things to check along the way:
> 
> 1) Make sure that the GRE interface on the squid box is seeing packets
> coming in. You should never see any packets going out the GRE
interface,
> and you will only see packets coming in the GRE interface after the
WCCP
> process on the router redirects them to the squid box.
> 
> 2) Keep in mind that you should separate the idea in your mind between
> WCCP traffic and tunneled HTTP traffic. The router and the squid
process
> talk to each other for status and service information with WCCP. HTTP
> traffic received from the client by the router for redirect is
> encapsulated in GRE (to preserve it) and then forwarded to the GRE
> interface.
> 
> 3) The global command "show ip wccp" is a useful router command. In
the
> output of this command, you should see two "Service Identifier"
sections
> (one for 80, one for 90 if you use my setup steps.) Within each
"Service
> Identifier" group, the number of service group clients and routers
> should each be 1. If they are not, them some facet of WCCP
conversation
> between the router and the squid server is not working. Check the
> iptables port 2048 setup step 20 part d above.
> 
> 4) wccp event debug commands are useful. Use "debug ip wccp events"
and
> set you router logging to debug level to see "Hello" and "Here_I_am"
> packets. You need to make sure you see both to insure that the router
> and squid box are WCCP talking.
> 
> 5) Surf the Web from a client and see if you get to a web site. If it
> doesn't work, check the items above, recheck the steps. 
> 
> It if does work, go to a site that tells you your IP to make sure
TPROXY
> is working and using the client IP and not the squid IP. It is
entirely
> possible that the whole setup will work, but not the client IP
spoofing.
> I also suggest that you burn the setup in with web surfing to make
sure
> it doesn't break.
> 
> I am interested to here people's feedback so that I can improve the
> steps above, as well as share optimizations.
> 
> Nick
> 
> 
> 
> 




[Index of Archives]     [Linux Audio Users]     [Samba]     [Big List of Linux Books]     [Linux USB]     [Yosemite News]

  Powered by Linux