[Unsnipped version of previous post follows]
On 21/12/13 12:53, Brian Burch wrote:
On 20/12/13 18:46, Brian Burch wrote:
On 20/12/13 13:05, Joel Gerber wrote:
Your problem is one I've come up against a number of times in the
past. You are definitely on the right route using "policy routing"
(otherwise known as utilizing IP Rules), but the snag you're hitting
is that the source address of a locally generated packet isn't chosen
until a destination route has already been selected. Using the from
statement in an ip rules table is only useful when you're routing
packets from other subnets.
What a quick and helpful reply! Not only did my problem motivate you to
respond, but I am very relieved to find the answer isn't just RTFM.
Your explanation about the timing for kernel selection of the source
address is crucial and now my failed experiments all make sense. Your
suggested solution also makes sense, although it is counter-intuitive...
at least to me!
Now, just so you know, the local table should normally be left alone.
It's used for locally connected subnets only. It is given the highest
priority on purpose... don't mess with it! :)
I read somewhere (possibly the howto guide or the ip-cref) a statement
that the kernel treats this table as read-only. My system is running
ubuntu 13.04 with a custom non-pae 3.8.0 kernel, so it is reasonably
modern. When I thought more about the purpose of this table, I decided
there would be no point in trying to modify it, even if that was
possible.
There are a few methods you could try here, but I'm only going to
recommend one of them. Have your main routing table hold the routes
that your local machine will use, and then have a separate table for
all routed hosts. Then, use ip rule statements to force any traffic
coming in from a routed host into that routing table.
IE: on firewall-router (assuming eth1 is facing your static subnet
hosts):
# ip rule list
0: from all lookup local
32765: iif eth1 lookup static-hosts
32766: from all lookup main
32767: from all lookup default
Have table main include all of the routes your local firewall-router
should use, and have static-hosts contain all of the routes that your
static-subnet-hosts should use.
Great! I understand the principle you propose and will try to implement
it over the weekend. I will update this thread when I have something to
report.
Excellent news!! My router/firewall is currently collecting 200+ updated
packages from the repository server!
The solution was achieved using ONLY policy routing, based on Joel's
suggested approach.
However, I hit some fairly nasty snags along the way and had to do a lot
of trial and error fiddling to make it work. I am certain my working
solution can be made more elegant, but that will take a couple of hours
pen-and-pencil time to map out the current routing table entries in
selection order and then perform some experiments.
I don't have time to do it at the moment, but will get started over the
next couple of days. If I get stuck, I will post the "current best"
configuration and ask for help to improve it.
I tried to improve my "current best" definitions, but my efforts to
remove apparent redundancy have all failed. Here is the setup:
brian@chenin:~$ ip address show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
state UNKNOWN qlen 1000
link/ether 00:15:8a:01:91:e3 brd ff:ff:ff:ff:ff:ff
inet 217.154.193.209/28 brd 217.154.193.223 scope global eth1
inet6 fe80::215:8aff:fe01:91e3/64 scope link
valid_lft forever preferred_lft forever
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
state UNKNOWN qlen 1000
link/ether 00:40:63:f5:a5:72 brd ff:ff:ff:ff:ff:ff
inet 172.16.101.2/24 brd 172.16.101.255 scope global eth0
inet6 fe80::240:63ff:fef5:a572/64 scope link
valid_lft forever preferred_lft forever
brian@chenin:~$ ip rule list
0: from all lookup local
32764: from 217.154.193.209 lookup CHENIN_TO_INTERNET
32765: from all iif eth1 lookup SUBNET_217
32766: from all lookup main
32767: from all lookup default
brian@chenin:~$ ip route list table CHENIN_TO_INTERNET
163.1.221.67 via 172.16.101.1 dev eth0
brian@chenin:~$ ip route list table SUBNET_217
default via 172.16.101.1 dev eth0
brian@chenin:~$ ip route list table main
163.1.221.67 dev eth0 scope link src 217.154.193.209
169.254.0.0/16 dev eth0 scope link metric 1000
172.16.101.0/24 dev eth0 proto kernel scope link src 172.16.101.2
217.154.193.208/28 dev eth1 proto kernel scope link src 217.154.193.209
I know Joel's suggestion did not include my CHENIN_TO_INTERNET rule, but
without it chenin simply times out on its http gets.
I also tried keeping CHENIN_TO_INTERNET, and removing the explicit route
in the main table, but that fails too, although I wasn't very surprised.
I have tried trimming these rules and routes to discover the minimum
requirement, but this seems to be the smallest working subset.
Here is my theory to account for what is happening:
1. The SUBNET_217 default route MUST be able to handle ANY traffic
arriving on eth1. When I changed the rule to match on just
217.154.193.208/28, the forwarded traffic became very glitchy, with poor
response times and some dropped tcp sessions.
2. The route to 163.1.221.67 in the main table is required to assign the
eth1 source address 217.154.193.209 to its outbound packets. This works
because the kernel cannot find any other matching routes from chenin to
163.1.221.67 (which might encourage it to assign eth0's 172.16.101.2 as
the source address).
4. Having assigned the 217.154.193.209 source address, the packet must
be routed to the next hop. The CHENIN_TO_INTERNET rule matches the
explicit source host address at a higher priority than the SUBNET_217
rule, so 172.16.101 is selected.
5. The main table has a route to 172.16.101.1 via eth0, so the packet is
sent on its way.
That seems to be three passes through the policy routing tables. Am I
correct?
I still feel it should be possible to improve the solution, but I don't
understand what is going on well enough to see how. I would be very
grateful if anyone can enlighten me.
Thanks,
Brian
Regards,
Brian
Once again, thanks for helping,
Brian
Joel Gerber
Network Specialist
Network Operations
Eastlink
E: Joel.Gerber@xxxxxxxxxxxxxxxx T: 519.786.1241
-----Original Message-----
From: lartc-owner@xxxxxxxxxxxxxxx [mailto:lartc-owner@xxxxxxxxxxxxxxx]
On Behalf Of Brian Burch
Sent: December-20-13 7:05 AM
To: Linux Advanced Routing
Subject: How to override the default source ipv4 address on packets
originating from a router
The mailing list has been dormant for 2 years, so I wonder whether
anyone is still listening for new questions?
My broadband router runs PPPoA and is dynamically assigned a single
ipv4 internet address by my ISP. I have a static subnet which I host
on a linux router/firewall (called chenin). The linux firewall and the
adsl router communicate via a non-internet-addressable private subnet.
Here is the topology:
Internet -- adsl-router-ppp0-ipv4-dynamic
-- adsl-router-eth0-172.16.101.1
--
-- firewall-router-eth0-172.16.101.2
-- firewall-router-217.154.193.209
--
-- static-subnet-hosts-217.154.193.154.208/28
Each of the hosts on the static subnet use 217.154.193.209 as their
own default route. The adsl router forwards all incoming packets to
the firewall/router's eth0. The firewall/router forwards all incoming
packets to the static subnet via its own eth1. The firewall/router
does not need to perform NAT, but it implements a simple set of
iptables rules for blacklisting, etc. /All this works perfectly./
My problem is that I need to download software updates (debian apt-get
http) for the firewall/router from a repository out on the internet.
The firewall/router can successfully ping the repo-server when I force
the source address like this:
ping -I 217.154.193.209 163.1.221.67
... but a simple "ping 163.1.221.67" (i.e. using the default source
address selection algorithm) fails. Wireshark confirms these
unanswered packets go out on eth0 with a source address of 172.16.101.2.
I believe I should be able to resolve this problem with iproute2
policy routing, but so far I have not been successful. I've tried
several variations, but they all give me the same "wrong" source
address.
Here is my simplest effort:
brian@chenin:~$ ip rule list
0: from all lookup local
32765: from 172.16.101.2 lookup CHENIN_ONLY
32766: from all lookup main
32767: from all lookup default
brian@chenin:~$ ip route list table CHENIN_ONLY
163.1.221.67 via 172.16.101.1 dev eth0 src 217.154.193.209
brian@chenin:~$ sudo ip route flush cache brian@chenin:~$ ip route get
163.1.221.67
163.1.221.67 via 172.16.101.1 dev eth0 src 172.16.101.2
cache
This shows me the source address in my policy rule has NOT been used,
or the routing table entry does not work the way I think.
The only rule table with higher priority than CHENIN_ONLY is local,
which contains routes for addresses local to the firewall/router -
nothing about remote addresses, i.e.
brian@chenin:~$ ip route list table local
broadcast 127.0.0.0 dev lo proto kernel scope link src 127.0.0.1
local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1
local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo proto kernel scope link src
127.0.0.1
broadcast 172.16.101.0 dev eth0 proto kernel scope link src
172.16.101.2
local 172.16.101.2 dev eth0 proto kernel scope host src 172.16.101.2
broadcast 172.16.101.255 dev eth0 proto kernel scope link src
172.16.101.2
broadcast 217.154.193.208 dev eth1 proto kernel scope link src
217.154.193.209
local 217.154.193.209 dev eth1 proto kernel scope host src
217.154.193.209
broadcast 217.154.193.223 dev eth1 proto kernel scope link src
217.154.193.209
and the main table looks like this:
brian@chenin:~$ ip route list table main
default via 172.16.101.1 dev eth0
169.254.0.0/16 dev eth0 scope link metric 1000
172.16.101.0/24 dev eth0 proto kernel scope link src 172.16.101.2
217.154.193.208/28 dev eth1 proto kernel scope link src
217.154.193.209
I wonder whether my unsuccessful "naked" pings to 163.1.221.67 are
passing through the tables three times:
1. My explicit policy should be applied to select a source address
of 217.154.193.209.
2. The main table is used to select the default route via 172.16.101.1:
3. The main table then provides the explicit route to the default
router, which seems to rewrite the source address to 172.16.101.2.
I have read the latest lartc HOWTO sections relevant to policy routing,
but didn't see anything similar to my situation. I also didn't see how
to get a verbose (i.e. blow by blow) output from the "ip route get"
command to show how it is selecting its route.
Am I trying to do the impossible here, or am I just making a mistake in
the way I am doing it?
I hope this strikes someone as an interesting question. If I can achieve
a solution, I would be happy to add the scenario to the HOWTO.
Brian
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe lartc" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
