you need to restore connmarks coming in from the wan so the system can
send them back out that way
this is the guts of my 4 way rig.
`iptables -t mangle -X`;
`iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark`;
`iptables -t mangle -A PREROUTING -i eth19 -j MARK --set-mark 0x01`;
`iptables -t mangle -A PREROUTING -i eth19 -j CONNMARK --save-mark`;
`iptables -t mangle -A PREROUTING -i eth20 -j MARK --set-mark 0x02`;
`iptables -t mangle -A PREROUTING -i eth20 -j CONNMARK --save-mark`;
`iptables -t mangle -A PREROUTING -i tun0 -j MARK --set-mark 0x03`;
`iptables -t mangle -A PREROUTING -i tun0 -j CONNMARK --save-mark`;
`iptables -t mangle -A PREROUTING -i tun1 -j MARK --set-mark 0x04`;
`iptables -t mangle -A PREROUTING -i tun1 -j CONNMARK --save-mark`;
`iptables -t mangle -A INPUT -i eth19 -j MARK --set-mark 0x01`;
`iptables -t mangle -A INPUT -i eth19 -j CONNMARK --save-mark`;
`iptables -t mangle -A INPUT -i eth20 -j MARK --set-mark 0x02`;
`iptables -t mangle -A INPUT -i eth20 -j CONNMARK --save-mark`;
`iptables -t mangle -A INPUT -i tun0 -j MARK --set-mark 0x03`;
`iptables -t mangle -A INPUT -i tun0 -j CONNMARK --save-mark`;
`iptables -t mangle -A INPUT -i tun1 -j MARK --set-mark 0x04`;
`iptables -t mangle -A INPUT -i tun1 -j CONNMARK --save-mark`;
`iptables -t mangle -A OUTPUT -j CONNMARK --restore-mark`;
`iptables -t mangle -A FORWARD -j CONNMARK --restore-mark`;
`iptables -t mangle -A OUTPUT -s $myeth19ip -j MARK --set-mark 0x01`;
`iptables -t mangle -A OUTPUT -s $myeth19ip -j CONNMARK --save-mark`;
`iptables -t mangle -A FORWARD -i eth19 -j MARK --set-mark 0x01`;
`iptables -t mangle -A FORWARD -i eth19 -j CONNMARK --save-mark`;
`iptables -t mangle -A OUTPUT -s $myeth20ip -j MARK --set-mark 0x02`;
`iptables -t mangle -A OUTPUT -s $myeth20ip -j CONNMARK --save-mark`;
`iptables -t mangle -A FORWARD -i eth20 -j MARK --set-mark 0x02`;
`iptables -t mangle -A FORWARD -i eth20 -j CONNMARK --save-mark`;
`iptables -t mangle -A OUTPUT -s 192.168.101.1 -j MARK --set-mark
0x03`;
`iptables -t mangle -A OUTPUT -s 192.168.101.1 -j CONNMARK
--save-mark`;
`iptables -t mangle -A FORWARD -i tun0 -j MARK --set-mark 0x03`;
`iptables -t mangle -A FORWARD -i tun0 -j CONNMARK --save-mark`;
`iptables -t mangle -A OUTPUT -s 192.168.102.1 -j MARK --set-mark
0x04`;
`iptables -t mangle -A OUTPUT -s 192.168.102.1 -j CONNMARK
--save-mark`;
`iptables -t mangle -A FORWARD -i tun1 -j MARK --set-mark 0x04`;
`iptables -t mangle -A FORWARD -i tun1 -j CONNMARK --save-mark`;
`iptables -t mangle -A POSTROUTING -o eth19 -j MARK --set-mark 0x1`;
`iptables -t mangle -A POSTROUTING -o eth19 -j CONNMARK --save-mark`;
`iptables -t mangle -A POSTROUTING -o eth20 -j MARK --set-mark 0x2`;
`iptables -t mangle -A POSTROUTING -o eth20 -j CONNMARK --save-mark`;
`iptables -t mangle -A POSTROUTING -o tun0 -j MARK --set-mark 0x3`;
`iptables -t mangle -A POSTROUTING -o tun0 -j CONNMARK --save-mark`;
`iptables -t mangle -A POSTROUTING -o tun1 -j MARK --set-mark 0x4`;
`iptables -t mangle -A POSTROUTING -o tun1 -j CONNMARK --save-mark`;
cheers
On 19/02/2012 9:40 AM, Lloyd Standish wrote:
Hi All,
I have a load balancing router to distribute traffic from an internal
LAN over several small (5 Mbit) uplinks, using NAT. The router works,
but I had a problem which prevents me from doing the balancing the way
I would like. I have spent weeks trying to fix the problem I will
describe below.
Here is an ASCII picture stolen from
http://lartc.org/howto/lartc.rpdb.multiple-links.html. This shows the
general scheme of my simple network setup (there are 5 uplinks instead
of 2). I have added the interface addresses to refer to below.
________
+------------+ /
|
gw1 | |
+-------------+
Provider 1 +-------
__ 192.168.1.7
| | /
___/ \_ +------+-------+
+------------+ |
_/ \__ | if1
| /
/ \ |
| |
| Local network -----+ Linux router | | Internet
\_ __/ |
| |
\__ __/ | if2
| \
\___/ +------+-------+
+------------+ |
200.91.104.144 |
gw2 | \
+-------------+
Provider 2 +-------
| | |
+------------+ \________
I have distinct routing tables for each interface, as described on
http://lartc.org/howto/lartc.rpdb.multiple-links.html.
The simple round-robin load-balancing described on this page takes
advantage of the _route_cache_ to choose a new connection based on
routing for a previous connection. This (partially, at least) solves
the primary problem with connmark-based load-balancing, which is the
tendency to break sessions.
Unfortunately I have a bad problem with round-robin balancing that I
have not been able to overcome: connections traveling through
interfaces having a private IP address (if1 above) are often broken.
Is there a known problem with this sort of load-balancing when there
is a private IP on the interface?
I'm quite sure the problem is not in the NAT done by Provider 1, since
when this same interface is used with my Linux router doing
connmark-based load balancing, connections are not dropped.
http://lartc.org/howto/lartc.rpdb.multiple-links.html explains this
simple routing scheme clearly, and I think I have followed it
carefully. Basically, for each interface I execute commands like
these (IP numbers and interfaces are replaced by variables from my
script, but it should be clear. CONNMARK<n> are simply chains to put
a fwmark on a packet. This is used only for special cases on this
router.):
ip route flush table $table
ip route add ${!network} dev ${interface} src ${!wan} table $table
ip route add ${!lan_net} dev ${lan_if} table $table
ip route add 127.0.0.0/8 dev lo table $table #ok
# also add route in main routing table for network (see below)
ip route add ${!network} dev ${interface} src ${!wan}
ip route add default via ${!gateway} dev ${interface} table $table
# the following rule is supposed to ensure packets are replied to over
the interface they came from
# frankly I don't clearly understand this; please comment
ip rule add from ${!wan} table $table priority
$((${#ifaces[@]}*100))
# masquerade outgoing connections on secondary interfaces
iptables -t nat -A POSTROUTING -o ${interface} -j SNAT
--to-source ${!wan}
# mark new incoming connection, *not* from LAN, so we route back out
the right interface
iptables -t mangle -A PREROUTING -i ${interface} -m state --state
NEW -j CONNMARK$((cardnum+1))
This is repeated for each interface, and then round-robin load
balancing is done over the interfaces by a command like this (this
example if for 3 interfaces):
ip route add default scope global nexthop via 192.168.1.1 dev eth1
weight 1 nexthop via 192.168.2.1 dev eth2 weight 1 nexthop via
200.91.104.144 dev ppp0 weight 1
Can you see anything wrong with this configuration? The problem is
very serious, since I am forced to use connmark to mark connections
for load-balancing, and the session-breaking is very annoying.
Regards,
Lloyd
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
