You're going to have to re-acquaint me with your network layout. I'm having a hard time remembering what all of your interfaces are being used for and how they fit into your network. Joel Gerber Network Specialist Network Operations Eastlink E: Joel.Gerber@xxxxxxxxxxxxxxxx T: 519.786.1241 -----Original Message----- From: lartc-owner@xxxxxxxxxxxxxxx [mailto:lartc-owner@xxxxxxxxxxxxxxx] On Behalf Of Brian Burch Sent: January-08-14 11:18 AM To: Linux Advanced Routing List Subject: Re: How to override the default source ipv4 address on packets originating from a router On 21/12/13 12:53, Brian Burch wrote: > On 20/12/13 18:46, Brian Burch wrote: >> On 20/12/13 13:05, Joel Gerber wrote: >>> Your problem is one I've come up against a number of times in the >>> past. You are definitely on the right route using "policy routing" >>> (otherwise known as utilizing IP Rules), but the snag you're hitting >>> is that the source address of a locally generated packet isn't >>> chosen until a destination route has already been selected. Using >>> the from statement in an ip rules table is only useful when you're >>> routing packets from other subnets. >> >> What a quick and helpful reply! Not only did my problem motivate you >> to respond, but I am very relieved to find the answer isn't just RTFM. >> >> Your explanation about the timing for kernel selection of the source >> address is crucial and now my failed experiments all make sense. Your >> suggested solution also makes sense, although it is counter-intuitive... >> at least to me! >> >>> Now, just so you know, the local table should normally be left alone. >>> It's used for locally connected subnets only. It is given the >>> highest priority on purpose... don't mess with it! :) >> >> I read somewhere (possibly the howto guide or the ip-cref) a >> statement that the kernel treats this table as read-only. My system >> is running ubuntu 13.04 with a custom non-pae 3.8.0 kernel, so it is >> reasonably modern. When I thought more about the purpose of this >> table, I decided there would be no point in trying to modify it, even >> if that was possible. >> >>> There are a few methods you could try here, but I'm only going to >>> recommend one of them. Have your main routing table hold the routes >>> that your local machine will use, and then have a separate table for >>> all routed hosts. Then, use ip rule statements to force any traffic >>> coming in from a routed host into that routing table. >>> >>> IE: on firewall-router (assuming eth1 is facing your static subnet >>> hosts): >>> >>> # ip rule list >>> 0: from all lookup local >>> 32765: iif eth1 lookup static-hosts >>> 32766: from all lookup main >>> 32767: from all lookup default >>> >>> Have table main include all of the routes your local firewall-router >>> should use, and have static-hosts contain all of the routes that >>> your static-subnet-hosts should use. >> >> Great! I understand the principle you propose and will try to >> implement it over the weekend. I will update this thread when I have >> something to report. > > Excellent news!! My router/firewall is currently collecting 200+ > updated packages from the repository server! > > The solution was achieved using ONLY policy routing, based on Joel's > suggested approach. > > However, I hit some fairly nasty snags along the way and had to do a > lot of trial and error fiddling to make it work. I am certain my > working solution can be made more elegant, but that will take a couple > of hours pen-and-pencil time to map out the current routing table > entries in selection order and then perform some experiments. > > I don't have time to do it at the moment, but will get started over > the next couple of days. If I get stuck, I will post the "current best" > configuration and ask for help to improve it. <snip> I tried to improve my "current best" definitions, but my efforts to remove apparent redundancy have all failed. Here is the setup: brian@chenin:~$ ip address show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000 link/ether 00:15:8a:01:91:e3 brd ff:ff:ff:ff:ff:ff inet 217.154.193.209/28 brd 217.154.193.223 scope global eth1 inet6 fe80::215:8aff:fe01:91e3/64 scope link valid_lft forever preferred_lft forever 3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000 link/ether 00:40:63:f5:a5:72 brd ff:ff:ff:ff:ff:ff inet 172.16.101.2/24 brd 172.16.101.255 scope global eth0 inet6 fe80::240:63ff:fef5:a572/64 scope link valid_lft forever preferred_lft forever brian@chenin:~$ ip rule list 0: from all lookup local 32764: from 217.154.193.209 lookup CHENIN_TO_INTERNET 32765: from all iif eth1 lookup SUBNET_217 32766: from all lookup main 32767: from all lookup default brian@chenin:~$ ip route list table CHENIN_TO_INTERNET 163.1.221.67 via 172.16.101.1 dev eth0 brian@chenin:~$ ip route list table SUBNET_217 default via 172.16.101.1 dev eth0 brian@chenin:~$ ip route list table main 163.1.221.67 dev eth0 scope link src 217.154.193.209 169.254.0.0/16 dev eth0 scope link metric 1000 172.16.101.0/24 dev eth0 proto kernel scope link src 172.16.101.2 217.154.193.208/28 dev eth1 proto kernel scope link src 217.154.193.209 I know Joel's suggestion did not include my CHENIN_TO_INTERNET rule, but without it chenin simply times out on its http gets. I also tried keeping CHENIN_TO_INTERNET, and removing the explicit route in the main table, but that fails too, although I wasn't very surprised. I have tried trimming these rules and routes to discover the minimum requirement, but this seems to be the smallest working subset. Here is my theory to account for what is happening: 1. The SUBNET_217 default route MUST be able to handle ANY traffic arriving on eth1. When I changed the rule to match on just 217.154.193.208/28, the forwarded traffic became very glitchy, with poor response times and some dropped tcp sessions. 2. The route to 163.1.221.67 in the main table is required to assign the eth1 source address 217.154.193.209 to its outbound packets. This works because the kernel cannot find any other matching routes from chenin to 163.1.221.67 (which might encourage it to assign eth0's 172.16.101.2 as the source address). 4. Having assigned the 217.154.193.209 source address, the packet must be routed to the next hop. The CHENIN_TO_INTERNET rule matches the explicit source host address at a higher priority than the SUBNET_217 rule, so 172.16.101 is selected. 5. The main table has a route to 172.16.101.1 via eth0, so the packet is sent on its way. That seems to be three passes through the policy routing tables. Am I correct? I still feel it should be possible to improve the solution, but I don't understand what is going on well enough to see how. I would be very grateful if anyone can enlighten me. Thanks, Brian -- To unsubscribe from this list: send the line "unsubscribe lartc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe lartc" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
