Hi,
can someone, please, give some advice about configuring multicast with
GFS? I know it might go out of topic, but it's perhaps useful for
others.
I'd use broadcast instead, but I have a problem that two groups of
servers sharing the same storage, but that are located in different
vlans, separated by router-on-a-stick, so I guess I have to use
multicast.
I've configured the router for multicast (config is right below), but it
doesn't seem to work.
Here's ascii pic of what I'm trying to make:
+--------+
| router |
+--------+
/ \
+----------+ +----------+
| switch A | | switch B |
| vlan 100 | | vlan 200 |
+----------+ +----------+
| |
+----------+ +----------+
| server A | | server B |
+----------+ +----------+
| |
+---------------------+
| san / storage |
+---------------------+
and relevant config (that I made this far):
router (cisco ios):
ip multicast-routing
!
interface FastEthernet0/0
description Branch A
ip address 1.1.1.1 255.255.255.0
ip pim sparse-dense-mode
ip igmp version 1
encapsulation dot1q 100
!
interface FastEthernet0/1
description Branch B
ip address 1.1.2.1 255.255.255.0
ip pim sparse-dense-mode
ip igmp version 1
encapsulation dot1q 200
!
ip pim send-rp-announce FastEthernet0/0 scope 16
ip pim send-rp-discovery scope 16
!
switch A and switch B are manageable Intel switches (dunno the exact
model; they are bundled with my IBM Blades), but have IGMP Snooping
turned on on every interface, and show default cisco pim groups
(224.0.0.40 and 224.0.0.39) on upstream ports.
/etc/cluster/cluster.conf for each cluster node is same (only important
part of config is here, ask for more, if needed):
<cluster name="test" config_version="1">
<cman>
<multicast addr="224.0.0.11"/>
</cman>
<nodes>
<node name="node1" votes="1">
<multicast addr="224.0.0.11" interface="eth0"/>
</node>
<node name="node2" votes="1">
<multicast addr="224.0.0.11" interface="eth0"/>
</node>
</nodes>
</cluster>
hosts ping each other, so networking part, as far as basic ip and
unicast is concerned, is working properly.
When starting, cman_tool says:
cluster # cman_tool join -d
multicast address 224.0.0.11
if eth0 for mcast address 224.0.0.11
setup up interface for address: node1
and as I can see from strace, "cman_tool join", this is what happens:
socket(0x1f /* PF_??? */, SOCK_DGRAM, 2) = 3
ioctl(3, 0x780b, 0x2) = 0
setsockopt(3, 0x2 /* SOL_?? */, 109, [6516590], 4) = 0
[...]
socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP) = 4
bind(4, {sa_family=AF_INET, sin_port=htons(6809), sin_addr=inet_addr("224.0.0.11")}, 16) = 0
socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP) = 5
bind(5, {sa_family=AF_INET, sin_port=htons(6809), sin_addr=inet_addr("1.1.1.2")}, 16) = 0
setsockopt(3, 0x2 /* SOL_?? */, 100, "\4\0\0\0\0\0\0\0", 8) = 0
setsockopt(3, 0x2 /* SOL_?? */, 103, "\5\0\0\0\0\0\0\0", 8) = 0
setsockopt(3, 0x2 /* SOL_?? */, 101, "\1\344\377\277\3\0\0\0\0\0\0\0\1\0\0\0smtp\0\'\1@\210\0"..., 36) = 0
close(3) = 0
exit_group(0) = ?
I've checked some programming examples on multicast as well as code for
cman, and I thing cman_tool/join.c has two problems:
- it never seems to issue setsockopt(..., IP_ADD_MEMBERSHIP...), thus,
never joins the group. I believe the problem is in if (!bcast) check,
which, if replaced with "if (bhe)" should work fine...
- it binds the socket with multicast address (fd = 4 in my case) instead
of local address. If the examples I looked are true, one should bind
local interface, and then specify mcast address in setsockopt call.
Can someone comment this issue? Am I going in completly wrong direction,
or multicast support isn't ready yet?
--
Lazar Obradovic, System Engineer
-----
laza@xxxxxx
YUnet International http://www.EUnet.yu
Dubrovacka 35/III, 11000 Belgrade
Tel: +381 11 3119901; Fax: +381 11 3119901
-----
This e-mail is confidential and intended only for the recipient.
Unauthorized distribution, modification or disclosure of its
contents is prohibited. If you have received this e-mail in error,
please notify the sender by telephone +381 11 3119901.
-----
