Hi All,
Currently, when using 8021q VLAN module to work on top of bonding, everything seems to work OK, but there are some issues that will not work according to our analysis. For example, any self-generated packets sent by bonding itself (e.g. arp-mon, TLB learning packets, ALB arp replies, etc.) do not have the VLAN id tag in them, and thus will not go through the switch. Also, in order to configure a VLAN interface, the underlying interface must be configured first to IP address 0.0.0.0. Since arp-mon uses bond's IP address, this might cause further problems. Other issue we've still not investigated, like what happens if bonding needs to parse a tagged packet for layer2/layer3 data, might still create more problems.
We have come up with some possible solution we would like to get comments on. First of all, our main guide line was not to duplicate code segments that are in the VLAN module and put them in bonding. Further, we figured bonding should not need to know about how the VLAN module handles hardware acceleration. On the other hand, bonding does need to know what VLAN tags are being used so it may send packets successfully through all the switch ports, so some kind of policy needs to be defined.
So here is what we've come up with until now.
1. Configuration Need to decide between: a. Block VLAN add/del operations when bond has no slaves. b. Block enslave/release of slaves when bond has no VLAN tags (needs a module parameter). c. Remove limitation of IP 0.0.0.0.
2. Indication Need to decide between: a. Add notification mechanism in VLAN module that bonding may register to listen to, and thus keep track of VLAN tags added/removed. b. Register to listen to net device register/unregister notifications to monitor creation/destruction of VLAN devices. Requires support for figuring out if a net device is a VLAN device, and also two vlan calls like get_realdev() and get_vlan_id() exported.
b) sounds good to me. There are flags that can let you know if it's a vlan device or not.
if.h:#define IFF_802_1Q_VLAN 0x1 /* 802.1Q VLAN device. */
c. Parse every packet going through bonding to collect VLAN tags.
3. Monitoring In order for bonding to be able to generate tagged packets on its own, two major changes need to be done. One is split the vlan_start_xmit function into insert_tag() and vlan_xmit(), so bonding may choose the required tag on its own, and let 8021q to the transmit. A second change is to split arp_send() into arp_create() and arp_send(), so bonding may pass all the usual parameters for arp creation, get a complete arp packet and then pass it to 8021q for tag insertion on transmission.
Hardware acceleration ===================== When coming to analyze what is required for adding support for VLAN hardware acceleration on top of bonding, other issues come to mind. Since add/del operations are defined and handshakes are performed between the VLAN module and the device driver, tracking VLAN tags is simpler and commands should just be propagated to the slaves. Enslaving/releasing slaves should also be simple and just require adding/removing existing VLAN tags from them. The problem is how to handle configuration issues.
I'd consider ignoring the HW accel unless you can prove it actually helps performance to a noticeable degree. I have never seen results of any benchmarking related to this...
1. Since adding the first VLAN tag requires some additional handshake, can bonding support that operation on a bond that already has slaves and is running? 2. What about removing the last tag from a bond? 3. Should the bond device declare itself as "VLAN challenged" before registering and remove that limitation only once it has slaves? 4. Should the bond declare itself as fully hardware acceleration capable to benefit from "strong" slaves while performing regular VLAN inserting/stripping for "weak" slaves? 5. How can bonding generate untagged packets and send them via hardware acceleration capable slaves (e.g. 802.3ad LACPDU) ?
-- Ben Greear <greearb@candelatech.com> <Ben_Greear AT excite.com> President of Candela Technologies Inc http://www.candelatech.com ScryMUD: http://scry.wanfear.com http://scry.wanfear.com/~greear
- : send the line "unsubscribe linux-net" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
