Dan,
there are two problems I see with current corosync.
1.) Localhost rebinding
2.) Rely on kernel multicast loop facility
My opinion on them is simple. Both must go and must be replaced by:
1.) Don't use multicast loop. Move message directly from send function
to receive function for local node
2.) Never rebind
It's really impossible for application authors to handle this "change of
identity" behavior.
And solution for both problems are on top place of my TODO list (so
expect them in 2.1 and backported to flatiron).
Regards,
Honza
dan clark napsal(a):
Hi Folks!
Thank you Christine for a well written test application and leading the way
with the apropos comment "NOTE: in reality we should also check the
nodeid". Some comments are more easily addressed then others!
During various failure tests a variety of conditions seem to trigger a
client application of the cpg library to change it's node identity. I know
this has been discussed under various guises with respect to the proper way
to fail a network (don't ifdown/ifup and interface. Oddly enough, however,
a common dynamic reconfiguration step on a node is to do a 'service network
restart' which tends to do ifdown/ifup on interfaces. Designing
applications to be resilient to common failures is often desirable,
including the restart of a service (such as corosync) so I have included a
slightly modified version of testcpg.c that provides such resiliency. I
wonder, however, if the nature of the changing identity of the node
information returned from cpg_local_get can be relied on across versions or
if this is abhorrent or transient behaviour that might change? Note, that
once the node identity has changed if an application continues to maintain
use of a group then once the cluster is reformed that group is isolated
from other groups, despite sharing a common name. Furthermore there are
impacts on other applications on the isolated node that might share the use
of that group.
On a separate note, is there a way to change the seemingly fixed 20-30
second delay upon the daemons re-joining a cluster separated due to
isolated network conditions (power cycling a switch for example)?
Note the following output indicating the first realization of a node that
the configuration has changed and how the local_get indicates that the node
identity is now (127.0.0.1) as opposed to the original value which was
(4.0.0.8). Tests performed on version 1.4.2.
dcube-8:28506 2012-04-09 14:02:06.803
ConfchgCallback: group 'GROUP'
Local node id is 127.0.0.1/100007f result 1
left node/pid 4.0.0.2/15796 reason: 3
nodes in group now 3
node/pid 4.0.0.4/2655
node/pid 4.0.0.6/4238
node/pid 4.0.0.8/28506
....
Finally even though the reported identity is loopback, the original id is
matched due to the static cache from the time of the join. Is there a race
condition, however, that just after the join if there is a network failure
that the identity might change before the initialization logic is complete
and thus even the modified sample program is open to a failure?
dcube-8:28506 2012-04-09 14:02:06.803
ConfchgCallback: group 'GROUP'
Local node id is 127.0.0.1/100007f result 1
left node/pid 4.0.0.8/28506 reason: 3
nodes in group now 0
We might have left the building pid 28506
We probably left the building switched identity? start nodeid 134217732
nodeid 134217732 current nodeid 16777343 pid 28506
We have left the building direct match start nodeid 134217732 nodeid
134217732 local get current nodeid 16777343 pid 28506
Perhaps the test application for the release could be updated to include
appropriate testing for the nodeid?
Dan
------------------------------------------------------------------------
_______________________________________________
discuss mailing list
discuss@xxxxxxxxxxxx
http://lists.corosync.org/mailman/listinfo/discuss
_______________________________________________
discuss mailing list
discuss@xxxxxxxxxxxx
http://lists.corosync.org/mailman/listinfo/discuss
