olivier arsac <oliv@xxxxxxxxx> wrote: [...] >Scenario: >your bond0 is running fine. it uses eth0 as active slave and eth2 as inactive >slave (different cards/ different driver to be safe) >some bozo reconfigures the switch port where your eth2 is plugged in and you >don't notice it (the crucial point here) >later on, your eth0 dies (or is unplugged by the brother of the first bozo) >and bamm... your nice HA node is off-line. >note: eth2 is still plugged and fine at the mii level. >even an arp would return OK so going from mii to arp for the bond is not the >right option. > >So. How could I check that an IP packet send via eth2 would really reach its >vlan? One obvious remedy is better bozo control, but for purposes of discussion, let's look at this as simply a question about assurance of the inactive path. The short answer to you question is that you can't do what you're trying to do. The bonding driver itself, as you've noted, is reasonably good at detecting link state, and connectivity to local network peers (via the ARP monitor), but doesn't provide full end-to-end path validation for all active and inactive slaves. The long answer is that end-to-end validation of the inactive path is fairly complicated, and can be tricky to do correctly. If an inactive slave transmits something, it may cause updating of forwarding tables on the network (either because ARP probes have this effect, or because many switches snoop traffic to determine which destination is reachable via which port), which is undesirable. Inactive slaves, at least in recent versions of bonding, also drop all incoming traffic to prevent the same packet from being delivered multiple times (if, e.g., a switch is flooding traffic to all ports of the bond for whatever reason). The ideal case is to issue, e.g., a ping (ICMP Echo Request) to some IP address on the desired destination. An IP-level probe is better in the grand scheme of things because an IP packet is routable and can reach off the local network (which an ARP cannot). If we move up to IPv6, this becomes more complicated, as the "inactive" slave would have to participated in the IPv6 stateless address autoconfiguration independently from the master, which also causes headaches for IPv6 snooping switches. So, to achieve an actual end to end test from the "inactive" slave to the peer of choice, it's necessary to isolate this traffic so that it properly returns to the "inactive" slave (and isn't routed back to the master). This separate communication needs to take place on a logically discrete network (which may also be physically discrete, as appears to be the case in your situation). If the "probe" network isn't separate from the "real" network, then intermediate routers may send the probes or replies over the wrong path, or improperly update forwarding tables and so on. The bonding driver today doesn't support this type of "independent" slave activity; all slaves are considered to be minions of the master, and aren't allowed to operate independently. I will point out that the ARP monitor, for a subset of cases, can come close to what you want. The current versions of the bonding driver support an "arp_validate" option, and can validate that the ARP probes (which are broadcasts) sent out over the active slave actually reach the inactive slave. It doesn't validate the ARP replies, as those are only received by the active slave, and it doesn't attempt to transmit on the inactive slave. >I tried a probably naive thing: >ifenslave -d bond0 eth2 >ifconfig eth2 $ip netmask 255.255.255.255 >route add -host $target eth2 >ping $target >(target is the gateway, ip is a reserved IP used only on this server to do that >check) >but it does not work as I hopped. Sometimes the ping is OK (but goes thru >bond0) sometimes it blocks... >The real question is How to do it properly (rather than how to fix my naive >try). I would hazard to guess that your problem here is likely one of routing. Maybe on the sender end, maybe on the reply end, maybe both. On the sender end, you can force ping to use a particular interface via the "-I" option, which will assure you that you're using the eth2 for the transmission. The question is going to be which path the reply packet takes to get back to the sender. The other problem, of course, is that you've removed your backup link from the bond, so if the primary should fail while you've got it running this ping test, you'll lose all connectivity. One perhaps cheap and easy, but not 100% reliable, method would be for you to periodically manually fail over the bond to whichever link is inactive (via ifenslave -c bond0 ethX, cycling X through your set of slave interfaces). The up side of this is that you'll exercise both paths regularly, so any bozo induced nonsense should become visible sooner rather than later; the precise interval for "sooner" depending upon how often the failover is induced. The down side is that a failover will probably lose at least a few packets, and you'll have to arrange your script or whatever to stop if you experience an actual failure. -J --- -Jay Vosburgh, IBM Linux Technology Center, fubar@xxxxxxxxxx _______________________________________________ LARTC mailing list LARTC@xxxxxxxxxxxxxxx http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
