Tony, [ snip ] : We hope to be able to route external internet web client packets : coming into an interface to go back out on the same interface. : The packets must go out the same interface they came in on [ snip ] : The interfaces are set up as: (eth0 is address 10.10.10.104, eth1 : is address 10.10.11.104). At first we thought we could use the : same subnet for each interface but after initial testing we : decided to simplify the situation by using different subnets. Since they are separate switches, you shouldn't have any problem with ARP flux if you want to use IPs inside the same subnet. But, simplifying the situation is probably a good idea. When dealing with complex situations such as the above, you may find that you wish to suppress some ARP replies. I have written a brief introduction to the issues involved with some links to other sources of docs [0]. Note, that only gets you around the (here possibly tricky, but probably not relevant) ARP concern. Skipping on to the fun stuff.... : 4: eth0: <BROADCAST,MULTICAST,PROMISC,UP> mtu 1500 qdisc pfifo_fast qlen 1000 : link/ether 00:0d:60:4e:33:d2 brd ff:ff:ff:ff:ff:ff : inet 10.10.10.104/24 brd 10.10.10.255 scope global eth0 : 5: eth1: <BROADCAST,MULTICAST,PROMISC,UP> mtu 1500 qdisc pfifo_fast qlen 1000 : link/ether 00:0d:60:4e:33:d3 brd ff:ff:ff:ff:ff:ff : inet 10.10.11.104/24 brd 10.10.11.255 scope global eth1 OK! : When we try these commands: : ip rule add iif eth0 prio 100 table 100 : ip route add default via 10.10.10.1 dev eth0 table 100 What this says to the kernel is (roughly): - if the packet to be routed has arrived inbound on eth0, then look up the route in route table 100 I'm guessing that this box is not a router, and therefore anything inbound on eth0 should be packets bound for a local destination. If that's the case, then this rule will never be traversed because rule 0, of the highest priority, will cause the kernel to use the 'local' routing table to be consulted, and the inbound packets will be routed to local destinations. So, the above two commands effectively do nothing. : ip rule add iif eth1 prio 200 table 200 : ip route add default via 10.10.11.2 dev eth1 table 200 : no packets are sent out of the interfaces. Same thing as above. [ snip ] : ip rule add from 10.10.12.3 prio 100 table 20 : ip route add default via 10.10.11.2 dev eth1 table 20 : we do not see traffic from 10.10.12.3 being answered on eth1, we also tried : 10.10.12.0/24 : and still no pings from 10.10.12.3 were answered. Any ideas why this rule : fails? Perhaps you have checked 'ip route show cache 10.10.12.3' to see where the packets were going? (Maybe out eth0? Maybe the only routes in the upstream to 10.10.12.3 are through eth1? Maybe the switch does its own reverse path filtering?) And speaking of rp_filter, have you checked the rp_filter sysctl on this box? [1] Also, : We tried with ip_forward set to zero and one (no difference), and : we are familiar with the use of the "ip route flush cache" : command when removing and adding routes. But, you don't really wish to use "ip_forward". It seems that you are running servers on these blades. You don't need/want them to be routers, do you? : Are we interpreting the documentation correctly for the ip rule : regarding the iif option? I don't think so. I think you are confusing TCP state with routing. Routing is a stateless affair. Indeed, Linux does have a route cache, and that means that a router will 'remember' a recently used route, but this is rather different from real statefulness in routing. It seems what you really wish is for the Linux box to keep track of flows and intelligently handle them. Fortunately, you don't really need this, given your description. You only need a way to convince packets addressed from a particular IP on your box to leave via a particular interface. Try this instead: echo 200 out_eth0 >> /etc/iproute2/rt_tables ip rule add from 10.10.10.104 table out_eth0 ip route add default via 10.10.10.1 table out_eth0 echo 201 out_eth1 >> /etc/iproute2/rt_tables ip rule add from 10.10.11.104 table out_eth1 ip route add default via 10.10.11.1 table out_eth1 I can't say that that my choice of routing table names are sensible, but they should give you the general idea. The table parameter takes a number or a name. If a name is supplied, the name is looked up in /etc/iproute2/rt_tables. : Can all source packets be routed out the same interface with : these commands? I don't really understand what this question asks. : Is there an easy solution for this using ip rule and ip route? While I wouldn't quite say easy, the should give you a start. To understand a bit better about Linux routing, I'd recommend Alexey Kuznetsov's iproute2 manual [2] and my own introduction [3]. Good luck! -Martin [0] http://linux-ip.net/html/ether-arp.html#ether-arp-flux [1] http://ipsysctl-tutorial.frozentux.net/chunkyhtml/theconfvariables.html#AEN634 [2] http://linux-ip.net/gl/ip-cref/ [3] http://linux-ip.net/html/routing-selection.html -- Martin A. Brown --- SecurePipe, Inc. --- mabrown@xxxxxxxxxxxxxx _______________________________________________ LARTC mailing list / LARTC@xxxxxxxxxxxxxxx http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/