[LARTC] Masq/route based on port

[Miron <miron@xxxxxxxx>]

I have following setup:

- eth0 is an internal network
- eth1 is an Internet connection (IP = 1.1.1.128, GW=1.1.1.1)
- eth2 is another Internet connection (IP = 2.2.2.128, GW=2.2.2.1)

I would like to masquerade port 80 through eth2, but all other traffic 
should be masq'ed through eth1.

My routing configuration:

   (default route in main table is 1.1.1.1)

   ip rule add fwmark 2 pref 1002 table 666

   ip route flush table 666
   ip route add default via 2.2.2.1 dev eth3 proto static table 666
   ip route flush cache

My firewall configuration:
   iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j MARK 
--set-mark 2
   iptables -t nat -A POSTROUTING -o eth1 -j SNAT --to-source 1.1.1.128
   iptables -t nat -A POSTROUTING -o eth2 -j SNAT --to-source 2.2.2.128

Unfortunately, this does not work.  Outgoing packets are fine.  Incoming 
packets on port 80 are not de-masqueraded and do not reach the internal 
hosts.

Also, if I change the ip rule above to be based on the source address 
(instead of a mark), connections start working fine.

Here is the output of 'ip rule ls', to prove that I do have fwmark compiled:
   0:      from all lookup local
   1002:   from all fwmark        2 lookup http
   32766:  from all lookup main
   32767:  from all lookup 253

I am wondering if there is some kind of bug related to the interaction 
between fwmark and NAT. Any ideas?

Thanks,
Miron Cuperman