Re: [LARTC] Masq/route based on port

Miron <miron@xxxxxxxx> · Fri, 07 Dec 2001 00:17:23 -0800

Okay, I figured it out. The secret is to set rp_filter to zero on the 
non-default interface.  I guess the kernel thought that the packets were 
source-routed because of the fact that the interface received packets it 
could not route.

That was frustrating, yet fun... ;)

Miron wrote:

I think we are on the same wavelength, except maybe exactly backwards. ;)

I'm trying to provide fast web browsing for clients that are on the 
internal network (talking to servers on the Internet).  Anything from 
the Internet with source port 80 should go through eth2.  Anything 
else should go through eth1.

I did some more investigation, and it seems that the Linux box is 
dropping all incoming packets arriving into eth2.  I can see packets 
with tcpdump, but the applications and NAT don't actually get it.  It 
must be getting dropped in the kernel.  It seems that this has 
something to do with the default route pointing to eth1.  I don't 
understand why the fact that the default route is not pointing to an 
interface should cause *incoming* packets to be dropped on that 
interface.

Sorry if my post was confusing.

Greg Scott wrote:

This is a home setup, not a server setup.  We have no servers on our 
network.  The reason we want port 80 on eth2 is because eth2 has 
more download bandwidth.  For other protocols we want eth1, because 
it has more symmetric bandwidth.

So anything that comes in from the Internet for port 80, no matter 
the source, you want the reply to go back out on ETH2.  And anything 
that

comes in from the Internet other than port 80, you want those replies to

go out ETH1.  So the web server process is inside your Linux box? 
Did I get that much right?  Or do I have it backwards? 
The Linux box is your internal LAN's default gateway and you want 
this box to decide which Internet interface to use, based on the 
destination port your internal client PCs choose? 
Hadn't thought about it that way before.

- Greg

-----Original Message-----
From: Miron [mailto:miron@xxxxxxxx]
Sent: Thursday, December 06, 2001 1:03 PM
To: Greg Scott
Cc: lartc@xxxxxxxxxxxxxxx
Subject: Re: [LARTC] Masq/route based on port

I have following setup:

- eth0 is an internal network
- eth1 is an Internet connection (IP = 1.1.1.128, GW=1.1.1.1)
- eth2 is another Internet connection (IP = 2.2.2.128, GW=2.2.2.1)

I would like to masquerade port 80 through eth2, but all other 
traffic should be masq'ed through eth1.

My routing configuration:

  (default route in main table is 1.1.1.1)

  ip rule add fwmark 2 pref 1002 table 666

  ip route flush table 666
  ip route add default via 2.2.2.1 dev eth3 proto static table 666
  ip route flush cache

My firewall configuration:

  iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK 
--set-mark 2

  iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128

  iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128

Unfortunately, this does not work.  Outgoing packets are fine.  
Incoming packets on port 80 are not de-masqueraded and do not reach 
the internal hosts.

Also, if I change the ip rule above to be based on the source 
address (instead of a mark), connections start working fine.

Here is the output of 'ip rule ls', to prove that I do have fwmark

compiled:

  0:      from all lookup local
  1002:   from all fwmark        2 lookup http
  32766:  from all lookup main
  32767:  from all lookup 253

I am wondering if there is some kind of bug related to the 
interaction between fwmark and NAT. Any ideas?

Thanks,
Miron Cuperman

Re: [LARTC] Masq/route based on port

Linux Advanced Routing and Traffic Control