I am running 2.4.20 with Julian Anastasov's patches (routes-2.4.20-9.diff), iptables v1.2.6a, iproute2-ss010824 on Debian. I have set up our internal gateway to multihome 2 T1's as described in http://www.linuxvirtualserver.org/~julian/nano.txt . The only difference is that I use multiple IP's on the external interfaces. EXA A.B.C.225 -------------------- A.B.C.226-230/27 eth0 GATEWAY ----------------- INT 192.168.0.1/24 eth1 -------------------- J.K.L.98-102/27 eth2 EXB J.K.L.97 1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo 2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100 link/ether 00:07:e9:cb:64:81 brd ff:ff:ff:ff:ff:ff inet A.B.C.226/27 brd A.B.C.255 scope global eth0 inet A.B.C.227/27 brd A.B.C.255 scope global secondary eth0:0 inet A.B.C.228/27 brd A.B.C.255 scope global secondary eth0:1 inet A.B.C.229/27 brd A.B.C.255 scope global secondary eth0:2 inet A.B.C.230/27 brd A.B.C.255 scope global secondary eth0:3 3: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100 link/ether 00:07:e9:cb:64:82 brd ff:ff:ff:ff:ff:ff inet 192.168.0.1/24 brd 192.168.0.255 scope global eth1 4: eth2: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100 link/ether 00:04:75:aa:c7:eb brd ff:ff:ff:ff:ff:ff inet J.K.L.98/27 brd J.K.L.127 scope global eth2 inet J.K.L.99/27 brd J.K.L.127 scope global secondary eth2:0 inet J.K.L.100/27 brd J.K.L.127 scope global secondary eth2:1 inet J.K.L.101/27 brd J.K.L.127 scope global secondary eth2:2 inet J.K.L.102/27 brd J.K.L.127 scope global secondary eth2:3 I use the following to set up routing and NAT (relevant excerpts shown). EXA_IF=eth0 EXA_NET=`ip addr ls $EXA_IF | awk '/inet/ { print $2 }' | head -1` EXA_IP=${EXA_NET//\/*/} EXA_GW=A.B.C.225 EXA_RNG=A.B.C.230 EXB_IF=eth2 EXB_NET=`ip addr ls $EXB_IF | awk '/inet/ { print $2 }' | head -1` EXB_IP=${EXB_NET//\/*/} EXB_GW=J.K.L.97 EXB_RNG=J.K.L.102 INT_IF=eth1 INT_NET=`ip addr ls $INT_IF | awk '/inet/ { print $2 }' | head -1` INT_IP=${INT_NET//\/*/} ip rule add prio 50 table main ip route del default table main 2>/dev/null ip rule add priority 100 from $EXB_NET table EXB ip route add default via $EXB_GW dev $EXB_IF src $EXB_IP \ proto static table EXB ip route append prohibit default table EXB metric 1 proto static ip rule add priority 150 from $EXA_NET table EXA ip route add default via $EXA_GW dev $EXA_IF src $EXA_IP \ proto static table EXA ip route append prohibit default table EXA metric 1 proto static # set up default route as multipath route ip rule add priority 200 table multi ip route add default table multi proto static \ nexthop via $EXB_GW dev $EXB_IF weight 1 \ nexthop via $EXA_GW dev $EXA_IF weight 1 ip route flush cache iptables -A FORWARD -m state \ --state ESTABLISHED,RELATED -j ACCEPT iptables -A INPUT -m state \ --state ESTABLISHED,RELATED -j ACCEPT iptables -A OUTPUT -m state \ --state ESTABLISHED,RELATED -j ACCEPT iptables -t nat -A PREROUTING -m state \ --state ESTABLISHED,RELATED -j ACCEPT iptables -t nat -A POSTROUTING -m state \ --state ESTABLISHED,RELATED -j ACCEPT iptables -t nat -A OUTPUT -m state \ --state ESTABLISHED,RELATED -j ACCEPT iptables -t nat -A POSTROUTING -s $INT_NET -o $EXB_IF \ -j SNAT --to-source $EXB_IP-$EXB_RNG iptables -t nat -A POSTROUTING -s $INT_NET -o $EXA_IF \ -j SNAT --to-source $EXA_IP-$EXA_RNG ( there is more but it is not relevant ) Here's the problem: Packets get lost along one of the routes. Everything runs smoothly when the EXB route is selected, but some packets get dropped when the EXA route is selected. For example: If I ping a host outside our network and the EXB route is chosen, I get the replies. On a bridge machine between the internal gateway and the EXB provider I can see that iptables is cycling through the range of SNAT ip's. If I ping a host outside out network and the EXA route is chosen, I get the replies sometimes and sometimes not. On the bridge machine I can see the packets go back and forth but they get discarded when they return to the internal gateway. I log my iptables drops and they don't show up anywhere. Inspection shows that the packets get lost when certain of the EXA range of ip's is chosen. It works when the external ip chosen is .226, .227, and .230 only. When the SNAT picks .228 or .229 it fails. Also, 2 identical outgoing routes appear in 'ip ro ls cache', but only one return route is created. When the EXB route is chosen I only see one outgoing and one incoming route. Any ideas? I've made sure that there are no other iptables rules to drop those packets. Arp looks fine. I can fix the problem by limiting the SNAT on EXA to one IP address and not using a range. When I do that everything works like it is supposed to. I'd like to be able to use a range of ip's though.
Attachment:
pgp00096.pgp
Description: PGP signature
