Hi Martin, First, just to clarify, all my packet destined for the internal net via our QSC provider was hitting the DNAT chain, and then lost. All packets going the other way was passing the FORWARD chain and then lost. So my problem seemed to be just after the incoming packet has arrived or before it leaves. I have tracked packets using extensive logging (and multiple changes in my firewall script) as well as tcpdump on all relevant interfaces in my system. The packet looks great when it comes in, correct information, etc. But never reaches the destination on the box. I have not played much with the firewall script after I got your reply, my ADSL died and I only just got it back up! But changing the rp_filter from one to zero seems to work, as I can track packets hitting my FORWARD chain now. So, right now I seem to be on the right track - I think the general problem with setting up something like this is that all relevant information is usually scattered over many places with mostly only fragments put together. But now now, Thanks Martin - you have been a great help :-) /Kim On Saturday 22 March 2003 06:57, Martin A. Brown wrote: > Kim, > > Well, I *think* the solution is a simple sysctl twiddle (see close to the > bottom), but I left my reasoning and research intact on the off-chance > that my method would benefit you or posterity reviewing this thread.. > > iptables mangle thoughts > - - - - - - - - - - - - - > I have line broken the iptables output, but I'm curious to know if you are > seeing any activity in these particular chains: > > # iptables -t mangle -nvL FORWARD > <snip> > 0 0 MARK all -- * eth1 0.0.0.0/0 > 0.0.0.0/0 MARK set 0x1 > 0 0 MARK all -- * * 212.202.108.160/27 \ > 0.0.0.0/0 MARK set 0x1 > 0 0 MARK all -- * * 0.0.0.0/0 \ > 212.202.108.160/27 MARK set 0x1 > </snip> > > I expect that the counter on the first of these three chains is going up, > because this is the chain through which all of your packets leaving eth1 > will pass. This might lead you to believe that the chain was performing > some useful function--but since the routing decision has already been > made, your packet is getting marked after the routing decision has been > made. I'm not sure about my diagnosis on this one, however.... > > I don't think the other two are doing anything. I think you > will probably want to use 192.168.1.160/27 here instead of the public > network IPs. > > Since your problem (for split access to the Internet via multiple > providers) is correctly routing the outbound packets, you really only need > to mark the outbound packets. You don't need to distinguish the inbound > packets because connection tracking will take care of restoring the > internal IP address. > > So, could you perform all of the marking in the PREROUTING chain? > > for internalnet in $ITX_NET $DMZ_NET ; do > iptables -t mangle -I PREROUTING -s $internalnet \ > -j MARK --set-mark $FWMARK > done > > I don't see why you'd choose the FORWARD chain in the mangle table for > this marking. I'd suggest the PREROUTING chain in the mangle table--at > least that way, you know the fwmark is available for ANY routing decision. > > : > Sounds like a routing problem. > > Sounds like I was speculating without enough information. :( Everything > looks perfect in your routing tables and RPDB! > > <snipped your routing tables and RPDB> > > : I'm also attaching the firewall script, which both generates the > : routing table and sets up the iptable rules. > > Ah, the joys of digging through mountains of iptables rules. At least > it's better than ipchains! Here's what I think you are seeing > > - outbound packet from 192.168.1.161 > - marked and rewritten to appear to the Internet from 212.202.108.161 > - ACK returns, accepted on eth1 > - connection tracking rewrites destination IP to 192.168.1.161 > - you lose packet! > > : I can see packets running out, and getting an ACK back. From logging I > : can see the packet getting into my DNAT, but then it stops! > > Do we concur? > > Question 1: Why do you say that the packet is hitting the DNAT? > Shouldn't the packet be handled by the connection tracking > mechanism? > Question 2: Did you try tcpdump'ing on eth2? Are packets showing up on > the wire with the wrong destination address? > Question 3: Did you try defining the minimal required iptables rules to > allow one single IP to reach the 'net via this split access > technique? > > After all of the above, I was reviewing your script one more time, and > discovered the rp_filter sysctl.... > > I think your rp_filter sysctl is disagreeing with your multiple uplink > paths. > > What happens when you change nothing of the above, and change this? > > # IP Spoof protection > if [ -f /proc/sys/net/ipv4/conf/all/rp_filter ]; then > echo 1 > /proc/sys/net/ipv4/conf/all/rp_filter > fi > > To > > # IP Spoof protection > if [ -f /proc/sys/net/ipv4/conf/all/rp_filter ]; then > for i in $( ls -1 /proc/sys/net/ipv4/conf ) ; do > echo 0 > /proc/sys/net/ipv4/conf/$i/rp_filter > done > fi > > Reverse path filtering is very handy in some situations as a very basic > antispoof measure, but when you have multiple paths to the same network, > it's important to turn off this spoof protection. > > For those following along on this thread, see the section on rp_filter in > the IP Sysctl tutorial: > > http://ipsysctl-tutorial.frozentux.net/ > http://ipsysctl-tutorial.frozentux.net/ipsysctl-tutorial.html#AEN616 > > Oskar Andreasson's cautionary note mocks me gently after an hour puzzling > about this problem! :) > > : I hope that you with this information can give me some hints. The net > : setup is 2 internal and two external. The rules should appear from the > : firewall script. > > I think this is what your network looks like: > > +------------------+ 212.202.108.160/27 +--------+ > 192.168.1.160/28 --+eth2 eth1+-- 192.168.0.0/24 ------+ router | > $DMZ_NET | | $QSC_NET +--------+ > > | linux | > | > | | +--------+ > > 192.168.1.176/28 --+eth3 eth0+-- 192.168.141.160/27 --+ router | > $ITX_NET +------------------+ (Arcor) +--------+ > > Good luck, Kim, > > -Martin
