Hello everybody,
I need a little help finding the correct combination of
arp_filter/rp_filter settings and routing.
I have a server with 2 NICs plugged into a layer-3 switch. Each nic is
on a seperate subnet (let's call the subnets 27 and 19, and the ips 27.1
and 19.1). The problem is that the switch gets it's arp entries messed
up so that both of the server's ips have the same MAC address (the MAC
address of the nic with the 19.1 ip). The causes the 27.1 ip to be
unreachable, except from the 27 subnet. As soon as the 27.1 arp entry
is deleted on the switch, it is repopulated with the correct entry and
everything works for a few minutes, until the arp entry gets replaced
with the 19.1 nic's MAC address again. I have tried to solve this with
a static arp entry in the switch, but a bug in my switch software
prevents permanent entries from sticking and my switch doesn't have
enough ram to run the latest software release.
What I think is happening is that the switch is sending out an arping,
first on the 27 subnet, and then on the 19 subnet. In response to the
first arping, the 27.1 nic responds with both the 27.1 ip and the 19.1
ip, overwriting the 19.1 nic's MAC address with the 27.1 nic's MAC
address. A very short time after, the reverse happens and the 27.1
nic's MAC address is overwritten with the 19.1 nic's MAC address. This
persists until the next arping cycle.
I can solve this problem by enabling rp_filter on both nics, but then I
have a problem with the default route. If if make the default route
through the 27 subnet, only the 27.1 ip works, if I make the default
route through the 19 subnet, only the 19.1 ip works. If I add a second
default route, it doesn't help (still only 1 ip is reachable). What I
really want is for all connections using 27.1 nic to use the 27.126
gateway and all connections using the 19.1 nic to use the 19.126
gateway. Is this possible?
Can someone sum up for exactly what arp_filter and rp_filter do?
Details:
Routing table from the server:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
x.x.19.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
x.x.27.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 x.x.19.126 0.0.0.0 UG 0 0 0 eth2
I have also tried this for the default route:
0.0.0.0 x.x.27.126 0.0.0.0 UG 0 0 0 eth1
eth1 and eth2 are Broadcom 5700 gigE nics (eth0 is onboard intel 10/100
and not used).
eth1 Link encap:Ethernet HWaddr 00:x:x:x:x:08
inet addr:x.x.27.1 Bcast:x.x.27.255 Mask:255.255.255.0
eth2 Link encap:Ethernet HWaddr 00:x:x:x:x:0D
inet addr:x.x.19.1 Bcast:x.x.19.255 Mask:255.255.255.0
The switch is an OmniStack 5024, running rev 4.1.4 GA.
-
: send the line "unsubscribe linux-net" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
