Just some quick answers to your questions inline below. (I've not had
time to consider this in detail sorry.)
On 2014-07-04 03:03, Nyamul Hassan wrote:
Thank you Amos & Eliezer for your responses!
Amos, we have enabled debug_options 11,2, but that did not show any
HTTP request being received by Squid, not even after doing the changes
that Eliezer suggested. But they did show up, when we reverted back
to "http_port 3127 intercept" related configuration. More details
below.
That is the problem then. Something is blocking the traffic arriving at
Squid listening port. selinux, rp_filter or ip_forward sysctl settings I
usually find are the problem for this, although there have been a few
cases where nobody could figure out why this was happening.
Eliezer, we tried with the "ip route add local default dev lo table
100", but still same problem. I think the wiki page
http://wiki.squid-cache.org/Features/Tproxy4 needs to be updated as it
clearly says "dev eth0" and not "dev lo".
see the /!\ notes under in the wiki page under the section about setting
up the route table.
The interface(s) to attach the table to is the one receiving the
packets. From your description I suspect you will have two interfaces -
one for each of Rtr1 and Rtr2.
For debugging try setting it for each interfaces receiving traffic and
see if TPROXY starts working.
Our setup would need a bit explanation. Please bear with me while I
describe as below:
For Traffic From Host:
#Start# Host (eth0 A.B.170.10/26) -->
--> (eth2 A.B.170.1/26) Rtr1 (eth2 A.B.170.1/26) -->
--> (eth0 A.B.170.24/26) SquidBox (eth1 A.B.169.21/28) -->
--> (eth2 A.B.169.17/28) Rtr2 (eth1 BGP peered uplink) -->
--> Internet #End#
For Traffic From Internet:
#Start# Internet -->
--> (eth1 BGP peered uplink) Rtr2 (eth2 A.B.169.17/28) -->
--> (eth1 A.B.169.21/28) SquidBox (eth0 A.B.170.24/28) -->
--> (eth0 A.B.170.10/26) Host #End#
* In my understanding, this should not pass through Rtr1 as as
SquidBox eth0 is in the same subnet as Host.
All HTTP traffic should pass through SquidBox, and in fact through Squid
itself. The TCP layer ports and packet serial numbers do not match what
the client is aware of, so any traffic accidentally reaching it without
going through Squid will be dropped.
Put this off to later though. Right now the packets are not even getting
into Squid.
Both Rtr1 & Rtr2 are Linux based router called Mikrotik, installed on
x86 hardware.
Rtr1 has the following rules:
/ip firewall mangle add action=mark-routing chain=prerouting
disabled=no dst-port=80 new-routing-mark=_to_squid_ passthrough=yes
protocol=tcp src-address=A.B.170.10
/ip route add disabled=no distance=1 dst-address=0.0.0.0/0
gateway=A.B.170.24 routing-mark=_to_squid_ scope=30 target-scope=10
Rtr2 has the following rules:
/ip firewall mangle add action=mark-routing chain=prerouting
disabled=no dst-address=A.B.170.10 new-routing-mark=_to_squid_
passthrough=yes protocol=tcp src-port=80
/ip route add disabled=no distance=1 dst-address=0.0.0.0/0
gateway=A.B.169.21 routing-mark=_to_squid_ scope=30 target-scope=10
The policy routing rules are the same on Rtr1 when we use the REDIRECT
rule in "iptables -t nat" for "http_port 3127 intercept", and in that
instance SquidBox works like a charm. All the HTTP request are now
showing up in "cache.log" because of "debug_options 11,2" as Amos
suggested.
Great. Thank you for these details. I am creating a Microtik wiki page
based on them.
However, whenever we remove the nat rules and introduce the mangle
rules + ip rule + ip route in table 100 for "http_port 3129 tproxy",
Rtr1 shows that the packets are marked and forwarded to SquidBox.
Even SquidBox properly logs the packets in "/var/log/messages" due to
the mangle table rule, but Squid process on SquidBox does not seem to
be receiving the packets. No HTTP request entry showing up in
"cache.log".
IPTables -L for mangle show the following:
[root@proxy01 ~]# iptables -vxnL --line-numbers -t mangle
Chain PREROUTING (policy ACCEPT 235 packets, 29632 bytes)
num pkts bytes target prot opt in out source
destination
1 0 0 ACCEPT all -- * * 0.0.0.0/0
A.B.169.21
2 6174 821596 ACCEPT all -- * * 0.0.0.0/0
A.B.170.24
3 1005 51367 ACCEPT all -- * * 0.0.0.0/0
A.B.174.0/24
4 0 0 ACCEPT all -- * * 0.0.0.0/0
M.N.0.66
5 49 3420 DIVERT tcp -- * * 0.0.0.0/0
0.0.0.0/0 socket
6 52 3840 LOG tcp -- * * 0.0.0.0/0
0.0.0.0/0 tcp dpt:80 LOG flags 0 level 4 prefix
`TProxy: '
7 52 3840 TPROXY tcp -- * * 0.0.0.0/0
0.0.0.0/0 tcp dpt:80 TPROXY redirect 0.0.0.0:3129
mark 0x1/0x1
The IP rule & route lists, and rt_tables & rp_filter show:
[root@proxy01 ~]# ip route list table squidtproxy
local default dev lo scope host
[root@proxy01 ~]# ip rule list
0: from all lookup local
32765: from all fwmark 0x1 lookup squidtproxy
32766: from all lookup main
32767: from all lookup default
[root@proxy01 ~]# cat /etc/iproute2/rt_tables
#
# reserved values
#
255 local
254 main
253 default
0 unspec
#
# local
#
#1 inr.ruhep
100 squidtproxy
[root@proxy01 ~]# find /proc/sys/net/ipv4/ -iname rp_filter
/proc/sys/net/ipv4/conf/all/rp_filter
/proc/sys/net/ipv4/conf/default/rp_filter
/proc/sys/net/ipv4/conf/lo/rp_filter
/proc/sys/net/ipv4/conf/eth0/rp_filter
/proc/sys/net/ipv4/conf/eth1/rp_filter
/proc/sys/net/ipv4/conf/gre0/rp_filter
/proc/sys/net/ipv4/conf/gretap0/rp_filter
[root@proxy01 ~]# find /proc/sys/net/ipv4/ -iname rp_filter -exec cat
{} +
0
0
0
0
0
0
0
Amos, we also looked into the "routing loop" that you mentioned.
Since there are two routers involved, Rtr1 & Rtr2, with Squid
connected to both of them, we used the rules above, so, Rtr1 only
policy-routes Host -> Squid and Rtr2 policy-routes Internet -> Squid.
This should be correct, no? In the very least, Squid should be
receiving the packets, and the HTTP request headers should show up in
"cache.log", shouldn't they?
That should be correct. And yes cache.log should at least be showing the
original client request. Even if the traffic were looping.
We apologize for the rather long email. Hopefully, you could
understand what our setup looks like, and help us in running Squid +
Tproxy.
With regards
HASSAN
On Thu, Jul 3, 2014 at 7:58 AM, Eliezer Croitoru <eliezer@xxxxxxxxxxxx>
wrote:
Hey There,
You have seem to use the wrong rules in ip route and maybe something
else.
I need more for the picture to understand what and how you implemented
it.
What I need is the IP and wires topology.
Wccp is not good for you(maybe) but the examples are perfect from any
aspect.
Take a peek at:
http://wiki.squid-cache.org/ConfigExamples/UbuntuTproxy4Wccp2
You can take the relevant rules from the article to correct yours.
basically what you need is:
#!/usr/bin/bash
echo "Loading modules.."
modprobe -a nf_tproxy_core xt_TPROXY xt_socket xt_mark ip_gre gre
LOCALIP="10.80.2.2"
echo "changing routing and reverse path stuff.."
for i in /proc/sys/net/ipv4/conf/*/rp_filter
do
echo 0 > $i
done
echo 1 > /proc/sys/net/ipv4/ip_forward
echo "creating routing table for tproxy..."
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
echo "creating iptables tproxy rules..."
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -p icmp -m icmp --icmp-type any -j ACCEPT
iptables -A FORWARD -i lo -j ACCEPT
iptables -t mangle -F
iptables -t mangle -A PREROUTING -d $LOCALIP -j ACCEPT
iptables -t mangle -N DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY
--tproxy-mark
0x1/0x1 --on-port 3129
##END OF FILE
The route towards the lo is important to enable the tproxy action.
In your settings I have seen that you have used something else which
will
probably cause some strange issues.
All The Bests,
Eliezer
On 07/03/2014 03:01 AM, Nyamul Hassan wrote:
Hi,
We are trying to run Squid 3.4.6 with TProxy. Earlier we used to run
Squid 2.7.Stable9 in "transparent" mode with a DNAT rule on the
router
box to redirect traffic. This being our first jibe at Squid3, we
have
successfully configured "intercept" mode with the router doing a
policy-based routing (instead of DNAT). All works quite well!
However, when we try to do a TProxy configuration, Squid does not
seem
to be seeing the traffic at all. Since Squid3 is working in
"intercept" we assume that is not the problem. IPTables is
configured
as follows:
*mangle
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DIVERT - [0:0]
-A PREROUTING -p tcp -m socket -j DIVERT
-A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "TProxy: "
-A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1
--on-port
3129
-A DIVERT -j MARK --set-mark 1
-A DIVERT -j ACCEPT
COMMIT
The Log option shows similar lines as follows (our IP omitted below):
Jul 3 05:15:24 proxy01 kernel: TProxy: IN=eth0 OUT=
MAC=00:22:4d:a7:9a:8c:00:15:17:c8:a0:39:08:00 SRC=<test>
DST=195.93.85.193 LEN=52 TOS=0x00 PREC=0x00 TTL=1 ID=25176 DF
PROTO=TCP SPT=3264 DPT=80 WINDOW=65535 RES=0x00 SYN URGP=0
We also tried both with and without the "IP" commands:
ip rule add fwmark 1 lookup 100
ip route add local default dev eth0 table 100
We have searched through Google, mailing lists, Squid Docs, but seems
like we are still missing through something. One thing though, a lot
of the TProxy examples accompany WCCP or Bridge. Are either of them
mandatory in TProxy setup? If not, could someone help us where we
are
doing things wrong?
Thanks in advance for youguidance.
Regards
HASSAN
