Still working on this problem, I have found what appears to be a bug.
The documentation seems to indicate that in a route definition if I have
"src x.x.x.x" it defines the outgoing IP address. I'm cautiously going
to say that doesn't seem to be the case.
I have these rules:
firewall2:root> ip rule show
0: from all lookup local
32765: from all fwmark 0x1 lookup mail.in
32766: from all lookup main
32767: from all lookup 253
which should (do) send marked packets to the ruleset "mail.in" (historic
name, not related to incoming mail). And I have these routes defined:
firewall2:root> ip route show
192.168.1.0/24 dev eth2 proto kernel scope link src 192.168.1.47
64.65.253.0/24 dev eth1 scope link
192.168.12.0/24 dev eth0 scope link
169.254.0.0/16 dev eth1 scope link
127.0.0.0/8 dev lo scope link
default via 192.168.1.1 dev eth2 src 192.168.1.47 metric 1
firewall2:root> ip route show table mail.in
default via 64.65.253.1 dev eth1 src 64.65.253.246
And if I run my multi-NIC tcpdump, I see that packets which are not
marked go out eth2, and those which ARE marked do in fact go out eth1 as
they should... but with the source IP of the default route, rather than
that specified in the mail.in definition. If I add
iptables -A POSTROUTING -t nat -o eth1 -m mark --mark 1 -j SNAT
--to 64.65.253.246
to the nat table, all of a sudden everything works. Note, this is the
simple two ISP case, running internal to my site, not the nasty one I
described originally (below).
So is the "src" portion of my table intended to set the source IP, or
did I misread the doc? And is that a bug or a feature?
Bill Davidsen wrote:
I've revisited this many times over the last five years, and always
wind up using BSD for my router. I need to set up another one, and I'd
really like to use Linux, so here's the problem.
This site has multiple default routes, in the sense that each can be
used for any external address. In order to deal with firewall rules
and bandwidth limitations, I need to be able to very selectively send
certain packets out certain interfaces, while allowing the fast/cheap
interface to handle anything else. This can't be done with traditional
routes, or I'm not smart enough to know how to handle sending via
different paths to the same IP depending on the port and protocol.
I had this working with the MARK target in the mangle table, and an
fwmark rule, but for some reason that wuit working in later 2.6
kernels. I want to use eth2 for default outbound, then have a lot of
special cases. If it were as simple as sending packets to machine A
via interface eth1 it would be easy, but for reasons of cost,
security, and politics I can't do that.
The firewall:
eth0 - private net 1
eth1 - T1 line, ISP1,static IP
eth2 - 5Mbit, ISP2, dhcp
eth3 - 768k DSL, ISP3, dhcp
Routing examples:
all smtp, pop3, imap - eth1
site JMinc, port ssh, eth1
site JMinc, UDP, eth2
site JMinc, port http, eth3
all other http, eth2
etc, etc, and many other complex etc
With iptables and the ROUTE target this would be easier, if Linux
routed packets out the appropriate interface based on source IP it
would be simple (SNAT), but there just doesn't seem to be a way to
short circuit routing logic and force a packet out a given interface
unconditionally. And multiple default routes seem to cause issues as
well.
New machine would be FC6 with all updates if I can find a way to get
the routing sane.
Any thoughts?
--
bill davidsen <davidsen@xxxxxxx>
CTO TMR Associates, Inc
Doing interesting things with small computers since 1979
-
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
