As an aside; I was using the DGS-3100 switches stacked. The new
generation of DGS-3120 switches I also used stacked, and are a *marked*
improvement over the 3100 series. I've not gone back to re-test the
other bond modes on these switches, as I must live within Red Hat's
supported configuration. However, this thread might just motivate me to
pull aside a test cluster and do some of my own testing again, on the
new switches.
Digimer
On 06/07/2012 04:51 AM, Radu Rendec wrote:
I also experimented with D-Link DGS-3xxx switches and the bonding
driver, but in a quite strange configuration: 2 distinct switches
without any "knowledge" of each other, and with each server having NIC
#1 connected in one switch and NIC #2 in the other.
In my case, the bonding driver actually splitted the traffic between the
2 links and I could achieve higher speeds than with a single link. This
is my config (nothing fancy):
[root@host ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond2
DEVICE=bond2
TYPE=Bonding
BONDING_OPTS="miimon=50"
ONBOOT=yes
BOOTPROTO=none
BRIDGE=br2
The bond interface was assigned to a bridge in my case, because I needed
to give network access to some VMs (besides the physical host).
If you don't need 802.1q tagging, I think you could try to create 2
VLANs in your switch and for all servers have one NIC in one vlan and
and another NIC in the other vlan and then a bond across those 2 NICs.
The drawback is that everything that is connected in this manner needs
to have exactly 2 NICs connected in the 2 VLANs. The other problem is
that if one NIC fails in one of the servers, it won't receive the
packets that are sent on the corresponding VLAN, so the server will not
receive half of the traffic that is meant for it.
I'm also curious about the results of your experiments. Please post back
if you have time.
Thanks,
Radu Rendec
On Thu, 2012-06-07 at 00:35 -0400, Digimer wrote:
I know that the only *supported* bond is Active/Passive (mode=1), which
of course provides no performance benefit.
I tested all types, using more modest D-Link DGS-3100 switches and all
other modes failed at some point in failure and recovery testing. If you
want to experiment, I'd suggest tweaking corosync's timeouts to be
(much?) more generous.
I'm curious to hear back on what your experimenting finds.
Digimer
On 06/07/2012 12:12 AM, Eric wrote:
I'm currently using the HP Procurve 2824 24-port Gigabit Ethernet switch
to for a backside network for synchronizing file systems between the
nodes in the group. Each host has 4 Gigabit NIC's and the goal is to
bond two of the Gigabit NIC's together to create a 2 Gbps link from any
host to any other host but what I'm finding is that the bonded links are
only capable of 1 Gbps from any host to any other host. Is it possible
to create a multi-Gigabit link between two hosts (without having to
upgrade to 10G) using a switch that "uses the SA/DA (Source
Address/Destination Address) method of distributing traffic across the
trunked links"?
The problem, at least as far as I can tell, comes down to the limitation
of ARP resolution (in the host) and mac-address tables (in the switch):
When configured to use Active Load Balancing, the kernel driver leaves
each of the interface's MAC addresses unchanged. In this scenario, when
Host A sends sends traffic to host Host B, the kernel uses the MAC
address of only one of Host B's NIC's as the DA. When the packet arrives
at the switch, the switch consults the mac-address table for the DA and
then sends the packet to the interface connected to the NIC with MAC
address equal to DA. Thus packets from Host A to Host B will only leave
the switch through one interface - the interface connected to the NIC
with MAC address equal to DA. This has the effect of limiting the
throughput from Host A to Host B to the speed of the one interface
connected to the NIC with MAC address equal to DA.
When configured to use IEEE 802.3ad (LACP), the kernel driver assigns
the same MAC address to all of the hosts' interfaces. In this scenario,
when Host A sends traffic to Host B, the kernel uses Host B's shared MAC
address as the DA. When the packet arrives at the switch, the switch
creates a hash based on the SA/DA pair, consults the mac-address table
for the DA, and and assigns the flow (i.e., traffic from Host A to Host
B) to one of the interfaces connected to Host B. Thus packets from Host
A to Host B will only leave the switch through one interface - the
interface determined by the SA/DA hash. This has the effect of limiting
the throughput from Host A to Host B to the speed of the one interface
determined by the hashing method. However, if the flow (from Host A to
Host B's shared MAC address) were to be distributed across the different
interfaces in a round-robin fashion (as the packets were leaving the
switch) the throughput between the hosts would equal the aggregate of
the links (IIUC).
Is this a limitation of the the Procurve's implementation of LACP? Do
other switches use different methods of distributing traffic across the
trunked links? Is there another method of aggregating the links between
the two hosts (e.g., multipathing)?
TIA,
Eric Pretorious
Truckee, CA
--
Linux-cluster mailing list
Linux-cluster@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/linux-cluster
--
Linux-cluster mailing list
Linux-cluster@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/linux-cluster
--
Digimer
Papers and Projects: https://alteeve.com
--
Linux-cluster mailing list
Linux-cluster@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/linux-cluster
