Dear Friends,
Thanks for your advise on my new cluster setup. I am going into failover testing of the cluster.
My configuraiton involves 3 active member and 1 standby servers and running Redhat 5.5 with Cluster installed.
In my active server, when I type "reboot", it can successfully relocate the service to the standby node.
However, If I type "init 0" to simulate server failure, it can't relocate to the standby node. Is it the correct behaviour?
Anything that I should check. Any advice?
Thanks for your help!
Below is the rgmanager log file from the standby server. It can detect the server down.
Jan 14 11:24:53 yzbstb01 clurgmgrd[16286]: <notice> Member 1 shutting down
Jan 14 11:25:17 yzbstb01 openais[15365]: [TOTEM] The token was lost in the OPERATIONAL state.
Jan 14 11:25:17 yzbstb01 openais[15365]: [TOTEM] Receive multicast socket recv buffer size (320000 bytes).
Jan 14 11:25:17 yzbstb01 openais[15365]: [TOTEM] Transmit multicast socket send buffer size (262142 bytes).
Jan 14 11:25:17 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from 2.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from 0.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] Saving state aru c8 high seq received c8
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] Storing new sequence id for ring 5e8
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] entering COMMIT state.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] entering RECOVERY state.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] position [0] member 10.10.10.164:
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1508 rep 10.10.10.164
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] aru c8 high delivered c8 received flag 1
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] position [1] member 10.10.10.165:
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1508 rep 10.10.10.164
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] aru c8 high delivered c8 received flag 1
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] position [2] member 10.10.10.167:
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1508 rep 10.10.10.164
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] aru c8 high delivered c8 received flag 1
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] Did not need to originate any messages in recovery.
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] CLM CONFIGURATION CHANGE
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] New Configuration:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.164)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.165)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.167)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] Members Left:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.166)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] Members Joined:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] CLM CONFIGURATION CHANGE
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] New Configuration:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.164)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.165)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.167)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] Members Left:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] Members Joined:
Jan 14 11:25:37 yzbstb01 openais[15365]: [SYNC ] This node is within the primary component and will provide service.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] entering OPERATIONAL state.
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] got nodejoin message 10.10.10.164
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] got nodejoin message 10.10.10.165
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] got nodejoin message 10.10.10.167
Jan 14 11:25:37 yzbstb01 openais[15365]: [CPG ] got joinlist message from node 4
Jan 14 11:25:37 yzbstb01 openais[15365]: [CPG ] got joinlist message from node 3
Jan 14 11:25:37 yzbstb01 openais[15365]: [CPG ] got joinlist message from node 2
Jan 14 11:51:34 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from 9.
Jan 14 11:51:37 yzbstb01 openais[15365]: [TOTEM] Saving state aru 25 high seq received 25
Jan 14 11:51:37 yzbstb01 openais[15365]: [TOTEM] Storing new sequence id for ring 5ec
Jan 14 11:51:37 yzbstb01 openais[15365]: [TOTEM] entering COMMIT state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] The token was lost in the COMMIT state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from 4.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] Storing new sequence id for ring 5f0
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering COMMIT state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from 13.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] Storing new sequence id for ring 5f4
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering COMMIT state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering RECOVERY state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] position [0] member 10.10.10.164:
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1512 rep 10.10.10.164
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] aru 25 high delivered 25 received flag 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] position [1] member 10.10.10.165:
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1512 rep 10.10.10.164
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] aru 25 high delivered 25 received flag 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] position [2] member 10.10.10.166:
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1512 rep 10.10.10.166
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] aru c high delivered c received flag 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] position [3] member 10.10.10.167:
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1512 rep 10.10.10.164
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] aru 25 high delivered 25 received flag 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] Did not need to originate any messages in recovery.
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] CLM CONFIGURATION CHANGE
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] New Configuration:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.164)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.165)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.167)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] Members Left:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] Members Joined:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] CLM CONFIGURATION CHANGE
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] New Configuration:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.164)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.165)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.166)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.167)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] Members Left:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] Members Joined:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.166)
Jan 14 11:51:47 yzbstb01 openais[15365]: [SYNC ] This node is within the primary component and will provide service.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering OPERATIONAL state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] got nodejoin message 10.10.10.165
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] got nodejoin message 10.10.10.166
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] got nodejoin message 10.10.10.167
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] got nodejoin message 10.10.10.164
Jan 14 11:51:47 yzbstb01 openais[15365]: [CPG ] got joinlist message from node 3
Jan 14 11:51:47 yzbstb01 openais[15365]: [CPG ] got joinlist message from node 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [CPG ] got joinlist message from node 2
Jan 14 11:51:47 yzbstb01 openais[15365]: [CPG ] got joinlist message from node 4
Jan 14 11:25:17 yzbstb01 openais[15365]: [TOTEM] The token was lost in the OPERATIONAL state.
Jan 14 11:25:17 yzbstb01 openais[15365]: [TOTEM] Receive multicast socket recv buffer size (320000 bytes).
Jan 14 11:25:17 yzbstb01 openais[15365]: [TOTEM] Transmit multicast socket send buffer size (262142 bytes).
Jan 14 11:25:17 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from 2.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from 0.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] Saving state aru c8 high seq received c8
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] Storing new sequence id for ring 5e8
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] entering COMMIT state.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] entering RECOVERY state.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] position [0] member 10.10.10.164:
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1508 rep 10.10.10.164
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] aru c8 high delivered c8 received flag 1
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] position [1] member 10.10.10.165:
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1508 rep 10.10.10.164
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] aru c8 high delivered c8 received flag 1
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] position [2] member 10.10.10.167:
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1508 rep 10.10.10.164
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] aru c8 high delivered c8 received flag 1
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] Did not need to originate any messages in recovery.
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] CLM CONFIGURATION CHANGE
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] New Configuration:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.164)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.165)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.167)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] Members Left:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.166)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] Members Joined:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] CLM CONFIGURATION CHANGE
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] New Configuration:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.164)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.165)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.167)
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] Members Left:
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] Members Joined:
Jan 14 11:25:37 yzbstb01 openais[15365]: [SYNC ] This node is within the primary component and will provide service.
Jan 14 11:25:37 yzbstb01 openais[15365]: [TOTEM] entering OPERATIONAL state.
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] got nodejoin message 10.10.10.164
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] got nodejoin message 10.10.10.165
Jan 14 11:25:37 yzbstb01 openais[15365]: [CLM ] got nodejoin message 10.10.10.167
Jan 14 11:25:37 yzbstb01 openais[15365]: [CPG ] got joinlist message from node 4
Jan 14 11:25:37 yzbstb01 openais[15365]: [CPG ] got joinlist message from node 3
Jan 14 11:25:37 yzbstb01 openais[15365]: [CPG ] got joinlist message from node 2
Jan 14 11:51:34 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from 9.
Jan 14 11:51:37 yzbstb01 openais[15365]: [TOTEM] Saving state aru 25 high seq received 25
Jan 14 11:51:37 yzbstb01 openais[15365]: [TOTEM] Storing new sequence id for ring 5ec
Jan 14 11:51:37 yzbstb01 openais[15365]: [TOTEM] entering COMMIT state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] The token was lost in the COMMIT state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from 4.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] Storing new sequence id for ring 5f0
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering COMMIT state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering GATHER state from 13.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] Storing new sequence id for ring 5f4
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering COMMIT state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering RECOVERY state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] position [0] member 10.10.10.164:
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1512 rep 10.10.10.164
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] aru 25 high delivered 25 received flag 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] position [1] member 10.10.10.165:
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1512 rep 10.10.10.164
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] aru 25 high delivered 25 received flag 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] position [2] member 10.10.10.166:
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1512 rep 10.10.10.166
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] aru c high delivered c received flag 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] position [3] member 10.10.10.167:
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] previous ring seq 1512 rep 10.10.10.164
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] aru 25 high delivered 25 received flag 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] Did not need to originate any messages in recovery.
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] CLM CONFIGURATION CHANGE
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] New Configuration:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.164)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.165)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.167)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] Members Left:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] Members Joined:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] CLM CONFIGURATION CHANGE
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] New Configuration:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.164)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.165)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.166)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.167)
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] Members Left:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] Members Joined:
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] r(0) ip(10.10.10.166)
Jan 14 11:51:47 yzbstb01 openais[15365]: [SYNC ] This node is within the primary component and will provide service.
Jan 14 11:51:47 yzbstb01 openais[15365]: [TOTEM] entering OPERATIONAL state.
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] got nodejoin message 10.10.10.165
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] got nodejoin message 10.10.10.166
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] got nodejoin message 10.10.10.167
Jan 14 11:51:47 yzbstb01 openais[15365]: [CLM ] got nodejoin message 10.10.10.164
Jan 14 11:51:47 yzbstb01 openais[15365]: [CPG ] got joinlist message from node 3
Jan 14 11:51:47 yzbstb01 openais[15365]: [CPG ] got joinlist message from node 1
Jan 14 11:51:47 yzbstb01 openais[15365]: [CPG ] got joinlist message from node 2
Jan 14 11:51:47 yzbstb01 openais[15365]: [CPG ] got joinlist message from node 4
-- Linux-cluster mailing list Linux-cluster@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/linux-cluster
