Hello,
Some time ago we were faced with a problem with multiple network
interfaces in the same subnetwork.
We would like to use several NIC's with IP addresses from the same
subnetwork with link redundancy functionality.
For example - we've got three NIC's at some host and each of them have
its separate IP addresses:
eth0: 11.0.0.1
eth1: 11.0.0.2
eth2: 11.0.0.3
from 11.0.0.0/24 subnet.
Now we would like to define some route for all of those interfaces like
that:
# ip route add 11.0.0.0/24 nexthop dev eth0 nexthop dev eth1 nexthop dev
eth2
As far as I know there is only one algorithm for handling multipath
routes in 2.6.33 kernel.
It doesn't care about physical link status. It uses randomization
combined with weights assigned for interfaces and provides some kind of
load balancing.
But we are not interested in load balancing. We are rather interested in
physical link redundancy.
Here there is how such solution can look like:
- we can define bunch of interfaces as the "nexthop dev" parameters and
assign unique priority to each of them
- we can use brand new algorithm which allows to select the highest
priority interface (with physical link up) and use it for communication.
For example if we have such a configuration:
eth0: prio 1 (highest priority) link up
eth1: prio 2 link up
eth2: prio 3 (lowest priority) link up
then the eth0 will be *the only interface* to reach other hosts from
11.0.0.0/24 subnet.
When the eth0 link becomes down - the eth1 will be used. Next if eth1
also becomes down - the eth2 will be used.
But if now eth0 becomes up - it will be selected as the default interface.
That is our imagination how it should work.
For testing purpose we've prepared some very simple patch for 2.6.33
kernel with this functionality.
It is available in the attachment.
It requires a few additional commands while configuring, for example:
# ip route add 11.0.0.0/24 nexthop dev eth0 nexthop dev eth1 nexthop dev
eth2
# echo 1 > /proc/sys/net/ipv4/route/nexthop_alg
# ip rule add from 11.0.0.1 table 1
# ip route add src 11.0.0.1 dev eth0 table 1
# ip rule add from 11.0.0.2 table 2
# ip route add src 11.0.0.2 dev eth1 table 2
# ip rule add from 11.0.0.3 table 3
# ip route add src 11.0.0.3 dev eth2 table 3
In example above the eth0 has the highest priority and the eth2 has the
lowest one.
With "echo" we're selecting the algorithm "1" which is the
implementation of algorithm described above.
Value "0" switches to the original kernel algorithm (this is also the
default value).
Depending on the physical link state of eth0, eth1 and eth2 it should
switch all of new connections from this host addressed to the
11.0.0.0/24 between eth0, eth1 and eth2 interfaces.
It concerns only the connections initiated from this host to 11.0.0.0/24
subnetwork.
The behavior of this algorithm can be observed using tcpdump - while
unplugging eth0 the whole network traffic to 11.0.0.0/24 should switch
from eth0 to eth1.
We know that this solution is far from being perfect. One of the
drawbacks is that it loses old TCP/IP connections established previously
on the link which has just gone down.
For example - we've established telnet connection with another host
(11.0.0.10) while the eth0 was selected as the highest priority interface.
This connection is working fine for some time and then the eth0 physical
link for some reasons becomes down.
At that moment the new algorithm from this patch selects next interface
with physical link up - and it will be eth1 in this case.
We are able to start new telnet connection with 11.0.0.10 (this time it
will goes through eth1), but we will not be able to use the old telnet
connection we established previously on eth0 interface. So - all the
connections established earlier via eth0 cannot be restored as long as
eth0 will be down.
It seems that this is not a fully-featured link redundancy solution, but
for now this is acceptable for us.
Keeping in mind the description above we would like to ask if there is
any other solution currently available and ready to use?
WK
--
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
