Hi people, I have two identical servers (both Sun X2100 M2's) that form part of a cluster of 3 machines (other machines will be added later). I want to bond two GB ethernet ports on these, which works perfectly on the one, but not on the other.
How can this be?
The one machine (named S2) detects no links up (with ethtool), yet the links are up. When I assign an ip to eth2 for instance, it works 100% despite ethtool claiming there is not link.
I understand that bonding uses ethtool to determine whether a link is up and then activates the bond. So how can I "fix" this?
How can this be?
The one machine (named S2) detects no links up (with ethtool), yet the links are up. When I assign an ip to eth2 for instance, it works 100% despite ethtool claiming there is not link.
I understand that bonding uses ethtool to determine whether a link is up and then activates the bond. So how can I "fix" this?
both machines have the following:
/etc/network/interfaces
# network interface settings
auto lo
iface lo inet loopback
auto eth2
iface eth2 inet static
auto eth3
iface eth3 inet static
iface eth0 inet manual
iface eth1 inet manual
auto bond0
iface bond0 inet manual
slaves eth2, eth3
bond_miimon 100
bond_mode 802.3ad
bond_xmit_hash_policy layer2
auto vmbr0
iface vmbr0 inet static
address 192.168.121.32
netmask 255.255.255.0
gateway 192.168.121.1
bridge_ports bond0
bridge_stp off
bridge_fd 0
# network interface settings
auto lo
iface lo inet loopback
auto eth2
iface eth2 inet static
auto eth3
iface eth3 inet static
iface eth0 inet manual
iface eth1 inet manual
auto bond0
iface bond0 inet manual
slaves eth2, eth3
bond_miimon 100
bond_mode 802.3ad
bond_xmit_hash_policy layer2
auto vmbr0
iface vmbr0 inet static
address 192.168.121.32
netmask 255.255.255.0
gateway 192.168.121.1
bridge_ports bond0
bridge_stp off
bridge_fd 0
And furthermore: /etc/udev/rules.d/70-persistent-net.rules
# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.
# PCI device 0x14e4:/sys/devices/pci0000:00/0000:00:0d.0/0000:05:00.0/0000:06:04.0 (tg3)
SUBSYSTEM=="net", ACTION="" DRIVERS=="?*", ATTR{address}=="00:16:36:76:0f:3d", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
# PCI device 0x14e4:/sys/devices/pci0000:00/0000:00:0d.0/0000:05:00.0/0000:06:04.1 (tg3)
SUBSYSTEM=="net", ACTION="" DRIVERS=="?*", ATTR{address}=="00:16:36:76:0f:3e", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"
# PCI device 0x10de:/sys/devices/pci0000:00/0000:00:09.0 (forcedeth)
SUBSYSTEM=="net", ACTION="" DRIVERS=="?*", ATTR{address}=="00:16:36:76:0f:40", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"
# PCI device 0x10de:/sys/devices/pci0000:00/0000:00:08.0 (forcedeth)
SUBSYSTEM=="net", ACTION="" DRIVERS=="?*", ATTR{address}=="00:16:36:76:0f:3f", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"
# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.
# PCI device 0x14e4:/sys/devices/pci0000:00/0000:00:0d.0/0000:05:00.0/0000:06:04.0 (tg3)
SUBSYSTEM=="net", ACTION="" DRIVERS=="?*", ATTR{address}=="00:16:36:76:0f:3d", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
# PCI device 0x14e4:/sys/devices/pci0000:00/0000:00:0d.0/0000:05:00.0/0000:06:04.1 (tg3)
SUBSYSTEM=="net", ACTION="" DRIVERS=="?*", ATTR{address}=="00:16:36:76:0f:3e", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"
# PCI device 0x10de:/sys/devices/pci0000:00/0000:00:09.0 (forcedeth)
SUBSYSTEM=="net", ACTION="" DRIVERS=="?*", ATTR{address}=="00:16:36:76:0f:40", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"
# PCI device 0x10de:/sys/devices/pci0000:00/0000:00:08.0 (forcedeth)
SUBSYSTEM=="net", ACTION="" DRIVERS=="?*", ATTR{address}=="00:16:36:76:0f:3f", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth3"
The MAC addresses correlate with the hardware.
The above is from the machine that works.
On the one that doesn't, the following:
/etc/network/interfaces
# network interface settings
auto lo
iface lo inet loopback
auto eth2
iface eth2 inet static
auto eth3
iface eth3 inet static
iface eth0 inet manual
iface eth1 inet manual
auto bond0
iface bond0 inet manual
slaves eth2, eth3
bond_miimon 100
bond_mode 802.3ad
bond_xmit_hash_policy layer2
auto vmbr0
iface vmbr0 inet static
address 192.168.121.31
netmask 255.255.255.0
gateway 192.168.121.1
bridge_ports bond0
bridge_stp off
bridge_fd 0
/etc/network/interfaces
# network interface settings
auto lo
iface lo inet loopback
auto eth2
iface eth2 inet static
auto eth3
iface eth3 inet static
iface eth0 inet manual
iface eth1 inet manual
auto bond0
iface bond0 inet manual
slaves eth2, eth3
bond_miimon 100
bond_mode 802.3ad
bond_xmit_hash_policy layer2
auto vmbr0
iface vmbr0 inet static
address 192.168.121.31
netmask 255.255.255.0
gateway 192.168.121.1
bridge_ports bond0
bridge_stp off
bridge_fd 0
The MAC addresses differ in the udev rules, but nothing else.
ethtool says eth2 and eth3 don't have a link.
On S2 (the working machine) it says eth2 is down and eth3 is up, but a bond is formed and the machine is connected.
What is happening here and how can it be resolved?
thanks
Roland
Roland
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
