I used bonding for High-Availability under linux.
See the attached little text (bonding-minihowto.txt) about what I'm
doing with...
I've just followed the bonding.txt documentation.
Almost everything works fine with a 2.4.18 kernel
There are some little problems about media status that can "setfault"
ifconfig
But since 2.4.19, it doesn't work at all :
When I assign an IP address on bond0 interface, the linux kernel freeze
!
I get the same problems when trying to apply bonding patches for 2.4.18
kernel.
Like the problem seems to in bonding and nothing else, I've just
retrieve old bonding versions from 2.4.18 original kernel :
- linux/drivers/net/bonding.c
- linux/include/linux/if_bonding.h
With this, everything work fine again !
So, problems are only in the modifications of bonding...
The main differences is the multicast support in bonding.
It's a bit strange, isn't it ?
Do someone has already seen this problem ?
Thanks in advance for your help...
-------------
Fabien SALVI Centre de Ressources Informatiques
Archamps, France -- http://www.cri74.org
PingOO GNU/linux distribution : http://www.pingoo.org
******** Using bonding Ethernet for a redundant (active/passive) ethernet link *******
Purpose: we want to have a High-Availability Cluster connected to 2 switches with ethernet redundancy
Each server is connected to 2 switches, but only one link is active, the other is usefull only
when there is a failure on the primary ethernet link.
1) Documentation :
See bonding.txt, included in linux kernel source in linux/Documentation/networking/bonding.txt
It's a mini-howto that explains how it works...
It's the only documentation of bonding.
The official homepage of the bonding project is (only patches for linux kernel):
http://sourceforge.net/projects/bonding/
You can find informations about MII from SCYLD website :
http://www.scyld.com/diag/mii-status.html (by Donald Becker)
2) Tools :
The only tool needed is included in linux kernel source at the same place (!!) as the bonding howto :
linux/Documentation/networking/ifenslave.c
You can compile it with :
gcc -Wall -Wstrict-prototypes -O -I/usr/src/linux/include ifenslave.c -o ifenslave
You must have selected as module "Bonding driver support" in the "Network device support" section when you compile the linux kernel.
3) Load module driver for NICs
example :
$ modprobe eepro100
You must use official linux drivers which support MII reporting concerning the link status.
With e100 or e1000 (intel drivers) bonding can't get status of links and you cannot use HA extensions...
4) Load bonding with options for HA :
example :
$ modprobe bonding miimon=200 mode=1
5) Put IP on bond0 interface :
example :
$ ifconfig bond0 195.202.0.6 netmask 255.255.255.128 broadcast 195.202.0.127 up
6) Add real interfaces to bond0 :
$ ifenslave bond0 eth0
$ ifenslave bond0 eth1
7) You should see something like that in kern.log :
kernel: bond0 registered with MII link monitoring set to 200 ms, in active-backup mode.
kernel: bond0: enslaving eth0 as a backup interface with a down link.
kernel: bond0: enslaving eth1 as an active interface with an up link.
8) Add default gateway :
$ route add default gw 195.202.0.1
9) Testing the bonding interface
9.1) ifconfig
* if the 2 interfaces are up :
$ ifconfig
bond0 Link encap:Ethernet HWaddr 00:02:B3:8B:96:9A
inet addr:195.202.0.6 Bcast:195.202.0.127 Mask:255.255.255.128
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:488812 errors:0 dropped:0 overruns:0 frame:2
TX packets:246895 errors:0 dropped:0 overruns:3 carrier:0
collisions:0 txqueuelen:0
RX bytes:714822873 (681.7 Mb) TX bytes:16330900 (15.5 Mb)
eth0 Link encap:Ethernet HWaddr 00:02:B3:8B:96:9A Media:unknown(auto)
inet addr:195.202.0.6 Bcast:195.202.0.127 Mask:255.255.255.128
UP BROADCAST NOTRAILERS RUNNING ALLMULTI SLAVE MTU:1500 Metric:1
RX packets:297754 errors:0 dropped:0 overruns:0 frame:2
TX packets:149869 errors:0 dropped:0 overruns:2 carrier:0
collisions:0 txqueuelen:100
RX bytes:433414739 (413.3 Mb) TX bytes:9921576 (9.4 Mb)
Interrupt:11 Base address:0xc000
eth1 Link encap:Ethernet HWaddr 00:02:B3:8B:96:9A
inet addr:195.202.0.6 Bcast:195.202.0.127 Mask:255.255.255.128
UP BROADCAST RUNNING NOARP SLAVE MULTICAST MTU:1500 Metric:1
RX packets:191058 errors:0 dropped:0 overruns:0 frame:0
TX packets:97026 errors:0 dropped:0 overruns:1 carrier:0
collisions:0 txqueuelen:100
RX bytes:281408134 (268.3 Mb) TX bytes:6409324 (6.1 Mb)
Interrupt:11 Base address:0xe000
Here, eth0 is the active link. eth1 has the "NOARP" flag.
* if eth1 become down, eth1 status change to this :
$ ifconfig eth1
eth1 Link encap:Ethernet HWaddr 00:02:B3:8B:96:9A
inet addr:195.202.0.6 Bcast:195.202.0.127 Mask:255.255.255.128
UP BROADCAST NOARP SLAVE MULTICAST MTU:1500 Metric:1
RX packets:191150 errors:0 dropped:0 overruns:0 frame:0
TX packets:97026 errors:0 dropped:0 overruns:1 carrier:0
collisions:0 txqueuelen:100
RX bytes:281413930 (268.3 Mb) TX bytes:6409324 (6.1 Mb)
Interrupt:11 Base address:0xe000
The "RUNNING" flag has disappeared
9.2) tcpdump
You can do tcpdump on bonding interface, you will see all traffic on the network.
You will capture nothing on enslave interfaces (eth0 or eth1)
9.3) Performance
We have the same performance than without bridge (real 10 MB/s if the switch is a good one).
The results are the same if the 2 links are up, ore only one is up...
9.4) ARP resolution
Only one interface of 2 (eth0 or eth1) is active in this HA configuration.
The bond0 interface will use an active interface and put the other interface in "backup mode".
In this mode, the interface doesn't answer to ARP requests.
The bond0 will take HWaddr from the first up interface added (it should always be eth0 it it's up).
It will keep it, even if the eth0 link is down.
9.5) Iptables firewall :
We can apply rules on the bridge ethernet.
For example, let's reject everything that come in input to veth0 for tcp/21 (FTP protocol) :
+ reject input for proto tcp and port 21 for interface veth0 :
$ iptables -A INPUT -p tcp -j REJECT -i bond0 --dport 21
+ status of iptables :
$ iptables -nvL
Chain INPUT (policy ACCEPT 176 packets, 11324 bytes)
pkts bytes target prot opt in out source destination
5 300 REJECT tcp -- bond0 * 0.0.0.0/0 0.0.0.0/0 tcp dpt:21 reject-with icmp-port-unreachable
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 99 packets, 9976 bytes)
pkts bytes target prot opt in out source destination
9.6) IP aliasing
IP aliasing works fine even with bonding interface, example :
$ ifconfig bond0:0 195.202.0.28 netmask 255.255.255.128 broadcast 195.202.0.127
$ ifconfig bond0:0
bond0:0 Link encap:Ethernet HWaddr 00:02:B3:8B:96:9A
inet addr:195.202.0.28 Bcast:195.202.0.127 Mask:255.255.255.128
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
10) Checking the redundant ethernet link:
- I "plug" eth0 link :
kernel: bond0: link status definitely up for interface eth0.
- I "unplug" eth1 link :
kernel: bond0: link status definitely down for interface eth1, disabling it and making interface eth0 the active one.
No interruption of traffic data...
The only problem is the delay on the switch when we plug the cable (around 20 seconds)
12) Unload bonding :
$ ifconfig bond0 down
$ rmmod bonding
- You will see this in kern.log :
kernel: bond0: releases all slaves
13) Automatic load of bonding :
Put this in modules.conf :
alias bond0 bonding
options bond0 miimon=200 mode=1
Or, for debian, create a file in /etc/modutils
