Kim,
Well, I *think* the solution is a simple sysctl twiddle (see close to the
bottom), but I left my reasoning and research intact on the off-chance
that my method would benefit you or posterity reviewing this thread..
iptables mangle thoughts
- - - - - - - - - - - - -
I have line broken the iptables output, but I'm curious to know if you are
seeing any activity in these particular chains:
# iptables -t mangle -nvL FORWARD
<snip>
0 0 MARK all -- * eth1 0.0.0.0/0
0.0.0.0/0 MARK set 0x1
0 0 MARK all -- * * 212.202.108.160/27 \
0.0.0.0/0 MARK set 0x1
0 0 MARK all -- * * 0.0.0.0/0 \
212.202.108.160/27 MARK set 0x1
</snip>
I expect that the counter on the first of these three chains is going up,
because this is the chain through which all of your packets leaving eth1
will pass. This might lead you to believe that the chain was performing
some useful function--but since the routing decision has already been
made, your packet is getting marked after the routing decision has been
made. I'm not sure about my diagnosis on this one, however....
I don't think the other two are doing anything. I think you
will probably want to use 192.168.1.160/27 here instead of the public
network IPs.
Since your problem (for split access to the Internet via multiple
providers) is correctly routing the outbound packets, you really only need
to mark the outbound packets. You don't need to distinguish the inbound
packets because connection tracking will take care of restoring the
internal IP address.
So, could you perform all of the marking in the PREROUTING chain?
for internalnet in $ITX_NET $DMZ_NET ; do
iptables -t mangle -I PREROUTING -s $internalnet \
-j MARK --set-mark $FWMARK
done
I don't see why you'd choose the FORWARD chain in the mangle table for
this marking. I'd suggest the PREROUTING chain in the mangle table--at
least that way, you know the fwmark is available for ANY routing decision.
: > Sounds like a routing problem.
Sounds like I was speculating without enough information. :( Everything
looks perfect in your routing tables and RPDB!
<snipped your routing tables and RPDB>
: I'm also attaching the firewall script, which both generates the
: routing table and sets up the iptable rules.
Ah, the joys of digging through mountains of iptables rules. At least
it's better than ipchains! Here's what I think you are seeing
- outbound packet from 192.168.1.161
- marked and rewritten to appear to the Internet from 212.202.108.161
- ACK returns, accepted on eth1
- connection tracking rewrites destination IP to 192.168.1.161
- you lose packet!
: I can see packets running out, and getting an ACK back. From logging I
: can see the packet getting into my DNAT, but then it stops!
Do we concur?
Question 1: Why do you say that the packet is hitting the DNAT?
Shouldn't the packet be handled by the connection tracking
mechanism?
Question 2: Did you try tcpdump'ing on eth2? Are packets showing up on
the wire with the wrong destination address?
Question 3: Did you try defining the minimal required iptables rules to
allow one single IP to reach the 'net via this split access
technique?
After all of the above, I was reviewing your script one more time, and
discovered the rp_filter sysctl....
I think your rp_filter sysctl is disagreeing with your multiple uplink
paths.
What happens when you change nothing of the above, and change this?
# IP Spoof protection
if [ -f /proc/sys/net/ipv4/conf/all/rp_filter ]; then
echo 1 > /proc/sys/net/ipv4/conf/all/rp_filter
fi
To
# IP Spoof protection
if [ -f /proc/sys/net/ipv4/conf/all/rp_filter ]; then
for i in $( ls -1 /proc/sys/net/ipv4/conf ) ; do
echo 0 > /proc/sys/net/ipv4/conf/$i/rp_filter
done
fi
Reverse path filtering is very handy in some situations as a very basic
antispoof measure, but when you have multiple paths to the same network,
it's important to turn off this spoof protection.
For those following along on this thread, see the section on rp_filter in
the IP Sysctl tutorial:
http://ipsysctl-tutorial.frozentux.net/
http://ipsysctl-tutorial.frozentux.net/ipsysctl-tutorial.html#AEN616
Oskar Andreasson's cautionary note mocks me gently after an hour puzzling
about this problem! :)
: I hope that you with this information can give me some hints. The net
: setup is 2 internal and two external. The rules should appear from the
: firewall script.
I think this is what your network looks like:
+------------------+ 212.202.108.160/27 +--------+
192.168.1.160/28 --+eth2 eth1+-- 192.168.0.0/24 ------+ router |
$DMZ_NET | | $QSC_NET +--------+
| linux |
| | +--------+
192.168.1.176/28 --+eth3 eth0+-- 192.168.141.160/27 --+ router |
$ITX_NET +------------------+ (Arcor) +--------+
Good luck, Kim,
-Martin
--
Martin A. Brown --- SecurePipe, Inc. --- mabrown@xxxxxxxxxxxxxx
