In the name of Allah Hello I use: Scientific Linux 6.0 Kernal version: 2.6.32-71.el6.x86_64 Corosync version: 1.4.2 I got corosync-1.4.2.tar.gz and compile and install it ./configure --disable-nss make make install igcc version: 4.4.4 Description: I configured corosync on some VMs that are connected with bridge and this is my corosync.conf file compatibility: none totem { i version: 2 token: 5000 token_retransmits_before_loss_const: 20 join: 1000 consensus: 7500 vsftype: none max_messages: 20 secauth: off threads: 0 nodeid: 1 clear_node_high_bit: yes interface { member { memberaddr: 192.168.0.100 } member { memberaddr: 192.168.0.101 } ringnumber: 0 bindnetaddr: 192.168.0.0 mcastaddr: 226.94.1.1 mcastport: 5405 ttl: 1 } } logging { fileline: off to_stderr: no to_logfile: yes to_syslog: yes logfile: /var/log/cluster/corosync.log debug: off timestamp: on logger_subsys { subsys: AMF debug: off } } amf { mode: disabled } When I start corosync after passing sometimes nodes starting to join and leave the cluster for a few times and then becomes all things normal and after passing sometimes this condition repeat. Under this condition all processes that are using corosync are killed I recieved these logs Mar 25 09:30:08 localhost corosync[3088]: [TOTEM ] A processor joined or left the membership and a new membership was forme d. Mar 25 09:30:08 localhost corosync[3088]: [CPG ] chosen downlist: sender r(0) ip(192.168.0.100) ; members(old:1 left:0) Mar 25 09:30:08 localhost corosync[3088]: [MAIN ] Completed service synchronization, ready to provide service. Mar 25 09:30:15 localhost corosync[3088]: [TOTEM ] A processor joined or left the membership and a new membership was forme d. Mar 25 09:30:15 localhost corosync[3088]: [CPG ] chosen downlist: sender r(0) ip(192.168.0.100) ; members(old:1 left:0) Mar 25 09:30:15 localhost corosync[3088]: [MAIN ] Completed service synchronization, ready to provide service. Mar 25 09:30:23 localhost corosync[3088]: [TOTEM ] A processor joined or left the membership and a new membership was forme d. Mar 25 09:30:23 localhost corosync[3088]: [CPG ] chosen downlist: sender r(0) ip(192.168.0.100) ; members(old:1 left:0) Mar 25 09:30:23 localhost corosync[3088]: [MAIN ] Completed service synchronization, ready to provide service. Mar 25 09:30:31 localhost corosync[3088]: [TOTEM ] A processor joined or left the membership and a new membership was forme d. Mar 25 09:30:31 localhost corosync[3088]: [CPG ] chosen downlist: sender r(0) ip(192.168.0.100) ; members(old:1 left:0) Mar 25 09:30:31 localhost corosync[3088]: [MAIN ] Completed service synchronization, ready to provide service. Mar 25 09:30:38 localhost corosync[3088]: [TOTEM ] A processor joined or left the membership and a new membership was forme d. Mar 25 09:30:38 localhost corosync[3088]: [CPG ] chosen downlist: sender r(0) ip(192.168.0.100) ; members(old:1 left:0) Mar 25 09:30:38 localhost corosync[3088]: [MAIN ] Completed service synchronization, ready to provide service. Mar 25 09:30:46 localhost corosync[3088]: [TOTEM ] A processor joined or left the membership and a new membership was forme d. Mar 25 09:30:46 localhost corosync[3088]: [CPG ] chosen downlist: sender r(0) ip(192.168.0.100) ; members(old:1 left:0) Mar 25 09:30:46 localhost corosync[3088]: [MAIN ] Completed service synchronization, ready to provide service. Mar 25 09:30:53 localhost corosync[3088]: [TOTEM ] A processor joined or left the membership and a new membership was forme d. Mar 25 09:30:53 localhost corosync[3088]: [CPG ] chosen downlist: sender r(0) ip(192.168.0.100) ; members(old:1 left:0) Mar 25 09:30:53 localhost corosync[3088]: [MAIN ] Completed service synchronization, ready to provide service. Mar 25 09:31:01 localhost corosync[3088]: [TOTEM ] A processor joined or left the membership and a new membership was forme d. Mar 25 09:31:01 localhost corosync[3088]: [CPG ] chosen downlist: sender r(0) ip(192.168.0.100) ; members(old:1 left:0) Mar 25 09:31:01 localhost corosync[3088]: [MAIN ] Completed service synchronization, ready to provide service. Mar 25 09:31:09 localhost corosync[3088]: [TOTEM ] A processor joined or left the membership and a new membership was formed I tested corosync on following conditions: 1. secauth: off 2. broadcast: yes 3. change mcastport 4. transport: udpu 5. On two physical nodes But the problem didn't solve The output of corosync-fplay is: Starting replay: head [28782] tail [0] rec=[1] Log Message=Corosync Cluster Engine ('1.4.2'): started and ready to provide service. rec=[2] Log Message=Corosync built-in features: nss rec=[3] Log Message=Successfully read main configuration file '/etc/corosync/corosync.conf'. rec=[4] Log Message=Token Timeout (5000 ms) retransmit timeout (247 ms) rec=[5] Log Message=token hold (187 ms) retransmits before loss (20 retrans) rec=[6] Log Message=join (1000 ms) send_join (0 ms) consensus (7500 ms) merge (200 ms) rec=[7] Log Message=downcheck (1000 ms) fail to recv const (2500 msgs) rec=[8] Log Message=seqno unchanged const (30 rotations) Maximum network MTU 1402 rec=[9] Log Message=window size per rotation (50 messages) maximum messages per rotation (20 messages) rec=[10] Log Message=missed count const (5 messages) rec=[11] Log Message=send threads (0 threads) rec=[12] Log Message=RRP token expired timeout (247 ms) rec=[13] Log Message=RRP token problem counter (2000 ms) rec=[14] Log Message=RRP threshold (10 problem count) rec=[15] Log Message=RRP multicast threshold (100 problem count) rec=[16] Log Message=RRP automatic recovery check timeout (1000 ms) rec=[17] Log Message=RRP mode set to none. rec=[18] Log Message=heartbeat_failures_allowed (0) rec=[19] Log Message=max_network_delay (50 ms) rec=[20] Log Message=HeartBeat is Disabled. To enable set heartbeat_failures_allowed > 0 rec=[21] Log Message=Initializing transport (UDP/IP Multicast). rec=[22] Log Message=Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0). rec=[23] Log Message=you are using ipc api v2 rec=[24] Log Message=Receive multicast socket recv buffer size (262142 bytes). rec=[25] Log Message=Transmit multicast socket send buffer size (262142 bytes). rec=[26] Log Message=The network interface [192.168.0.63] is now up. rec=[27] Log Message=Created or loaded sequence id bf34.192.168.0.63 for this ring. rec=[28] Log Message=Service failed to load 'pacemaker'. rec=[29] Log Message=Service engine loaded: corosync extended virtual synchrony service rec=[30] Log Message=Service engine loaded: corosync configuration service rec=[31] Log Message=Service engine loaded: corosync cluster closed process group service v1.01 rec=[32] Log Message=Service engine loaded: corosync cluster config database access v1.01 rec=[33] Log Message=Service engine loaded: corosync profile loading service rec=[34] Log Message=Service engine loaded: corosync cluster quorum service v0.1 rec=[35] Log Message=Compatibility mode set to whitetank. Using V1 and V2 of the synchronization engine. rec=[36] Log Message=entering GATHER state from 15. rec=[37] Log Message=Creating commit token because I am the rep. rec=[38] Log Message=Saving state aru 0 high seq received 0 rec=[39] Log Message=Storing new sequence id for ring bf38 rec=[40] Log Message=entering COMMIT state. rec=[41] Log Message=got commit token rec=[42] Log Message=entering RECOVERY state. rec=[43] Log Message=position [0] member 192.168.0.63: rec=[44] Log Message=previous ring seq bf34 rep 192.168.0.63 rec=[45] Log Message=aru 0 high delivered 0 received flag 1 rec=[46] Log Message=Did not need to originate any messages in recovery. rec=[47] Log Message=got commit token rec=[48] Log Message=Sending initial ORF token rec=[49] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0 rec=[50] Log Message=install seq 0 aru 0 high seq received 0 rec=[51] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 rec=[52] Log Message=install seq 0 aru 0 high seq received 0 rec=[53] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 rec=[54] Log Message=install seq 0 aru 0 high seq received 0 rec=[55] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 rec=[56] Log Message=install seq 0 aru 0 high seq received 0 rec=[57] Log Message=retrans flag count 4 token aru 0 install seq 0 aru 0 0 rec=[58] Log Message=Resetting old ring state rec=[59] Log Message=recovery to regular 1-0 rec=[60] Log Message=Delivering to app 1 to 0 rec=[61] Log Message=This node is within the primary component and will provide service. rec=[62] Log Message=entering OPERATIONAL state. rec=[63] Log Message=A processor joined or left the membership and a new membership was formed. rec=[64] Log Message=entering GATHER state from 11. rec=[65] Log Message=got commit token rec=[66] Log Message=Saving state aru 0 high seq received 0 rec=[67] Log Message=Storing new sequence id for ring bf3c rec=[68] Log Message=entering COMMIT state. rec=[69] Log Message=got commit token rec=[70] Log Message=entering RECOVERY state. rec=[71] Log Message=TRANS [0] member 192.168.0.63: rec=[72] Log Message=position [0] member 192.168.0.59: rec=[73] Log Message=previous ring seq bf38 rep 192.168.0.59 rec=[74] Log Message=aru 24 high delivered 24 received flag 1 rec=[75] Log Message=position [1] member 192.168.0.63: rec=[76] Log Message=previous ring seq bf38 rep 192.168.0.63 rec=[77] Log Message=aru 0 high delivered 0 received flag 1 rec=[78] Log Message=position [2] member 192.168.0.68: rec=[79] Log Message=previous ring seq bf38 rep 192.168.0.59 rec=[80] Log Message=aru 24 high delivered 24 received flag 1 rec=[81] Log Message=position [3] member 192.168.0.70: rec=[82] Log Message=previous ring seq bf38 rep 192.168.0.59 rec=[83] Log Message=aru 24 high delivered 24 received flag 1 rec=[84] Log Message=Did not need to originate any messages in recovery. rec=[85] Log Message=mcasted message added to pending queue rec=[86] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru ffffffff rec=[87] Log Message=install seq 0 aru 0 high seq received 0 rec=[88] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 rec=[89] Log Message=install seq 0 aru 0 high seq received 0 rec=[90] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 rec=[91] Log Message=install seq 0 aru 0 high seq received 0 rec=[92] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 rec=[93] Log Message=install seq 0 aru 0 high seq received 0 rec=[94] Log Message=retrans flag count 4 token aru 0 install seq 0 aru 0 0 rec=[95] Log Message=Resetting old ring state rec=[96] Log Message=recovery to regular 1-0 rec=[97] Log Message=Delivering to app 1 to 0 rec=[98] Log Message=This node is within the primary component and will provide service. rec=[99] Log Message=entering OPERATIONAL state. rec=[100] Log Message=A processor joined or left the membership and a new membership was formed. rec=[101] Log Message=mcasted message added to pending queue rec=[102] Log Message=Delivering 0 to 2 rec=[103] Log Message=Delivering MCAST message with seq 1 to pending delivery queue rec=[104] Log Message=confchg entries 4 rec=[105] Log Message=Delivering MCAST message with seq 2 to pending delivery queue rec=[106] Log Message=confchg entries 4 rec=[107] Log Message=Barrier Start Received From 1 rec=[108] Log Message=Barrier completion status for nodeid 2 = 0. rec=[109] Log Message=Barrier completion status for nodeid 1 = 1. rec=[110] Log Message=Barrier completion status for nodeid 3 = 0. rec=[111] Log Message=Barrier completion status for nodeid 4 = 0. rec=[112] Log Message=Received ringid(192.168.0.59:48956) seq 1 rec=[113] Log Message=Received ringid(192.168.0.59:48956) seq 2 rec=[114] Log Message=Received ringid(192.168.0.59:48956) seq 3 rec=[115] Log Message=Delivering 2 to 3 rec=[116] Log Message=Delivering MCAST message with seq 3 to pending delivery queue rec=[117] Log Message=confchg entries 4 rec=[118] Log Message=Barrier Start Received From 3 rec=[119] Log Message=Barrier completion status for nodeid 2 = 0. rec=[120] Log Message=Barrier completion status for nodeid 1 = 1. rec=[121] Log Message=Barrier completion status for nodeid 3 = 1. rec=[122] Log Message=Barrier completion status for nodeid 4 = 0. rec=[123] Log Message=Received ringid(192.168.0.59:48956) seq 4 rec=[124] Log Message=Delivering 3 to 4 rec=[125] Log Message=Delivering MCAST message with seq 4 to pending delivery queue rec=[126] Log Message=confchg entries 4 rec=[127] Log Message=Barrier Start Received From 4 rec=[128] Log Message=Barrier completion status for nodeid 2 = 0. rec=[129] Log Message=Barrier completion status for nodeid 1 = 1. rec=[130] Log Message=Barrier completion status for nodeid 3 = 1. rec=[131] Log Message=Barrier completion status for nodeid 4 = 1. rec=[132] Log Message=Received ringid(192.168.0.59:48956) seq 5 rec=[133] Log Message=Delivering 4 to 5 rec=[134] Log Message=Delivering MCAST message with seq 5 to pending delivery queue rec=[135] Log Message=confchg entries 4 rec=[136] Log Message=Barrier Start Received From 2 rec=[137] Log Message=Barrier completion status for nodeid 2 = 1. rec=[138] Log Message=Barrier completion status for nodeid 1 = 1. rec=[139] Log Message=Barrier completion status for nodeid 3 = 1. rec=[140] Log Message=Barrier completion status for nodeid 4 = 1. rec=[141] Log Message=Synchronization barrier completed rec=[142] Log Message=Synchronization actions starting for (dummy CLM service) rec=[143] Log Message=Received ringid(192.168.0.59:48956) seq 6 rec=[144] Log Message=Delivering 5 to 6 rec=[145] Log Message=Delivering MCAST message with seq 6 to pending delivery queue rec=[146] Log Message=mcasted message added to pending queue rec=[147] Log Message=releasing messages up to and including 5 rec=[148] Log Message=Delivering 6 to 7 rec=[149] Log Message=Delivering MCAST message with seq 7 to pending delivery queue rec=[150] Log Message=Received ringid(192.168.0.59:48956) seq 7 rec=[151] Log Message=Received ringid(192.168.0.59:48956) seq 8 rec=[152] Log Message=Delivering 7 to 8 rec=[153] Log Message=Delivering MCAST message with seq 8 to pending delivery queue rec=[154] Log Message=Received ringid(192.168.0.59:48956) seq 9 rec=[155] Log Message=Delivering 8 to 9 rec=[156] Log Message=Delivering MCAST message with seq 9 to pending delivery queue rec=[157] Log Message=Received ringid(192.168.0.59:48956) seq a rec=[158] Log Message=Delivering 9 to a rec=[159] Log Message=Delivering MCAST message with seq a to pending delivery queue rec=[160] Log Message=confchg entries 4 rec=[161] Log Message=Barrier Start Received From 2 rec=[162] Log Message=Barrier completion status for nodeid 2 = 1. rec=[163] Log Message=Barrier completion status for nodeid 1 = 0. rec=[164] Log Message=Barrier completion status for nodeid 3 = 0. rec=[165] Log Message=Barrier completion status for nodeid 4 = 0. rec=[166] Log Message=mcasted message added to pending queue rec=[167] Log Message=releasing messages up to and including 6 rec=[168] Log Message=Delivering a to b rec=[169] Log Message=Delivering MCAST message with seq b to pending delivery queue rec=[170] Log Message=confchg entries 4 rec=[171] Log Message=Barrier Start Received From 1 rec=[172] Log Message=Barrier completion status for nodeid 2 = 1. rec=[173] Log Message=Barrier completion status for nodeid 1 = 1. rec=[174] Log Message=Barrier completion status for nodeid 3 = 0. rec=[175] Log Message=Barrier completion status for nodeid 4 = 0. rec=[176] Log Message=Received ringid(192.168.0.59:48956) seq b rec=[177] Log Message=Received ringid(192.168.0.59:48956) seq c rec=[178] Log Message=Delivering b to c rec=[179] Log Message=Delivering MCAST message with seq c to pending delivery queue rec=[180] Log Message=confchg entries 4 rec=[181] Log Message=Barrier Start Received From 3 rec=[182] Log Message=Barrier completion status for nodeid 2 = 1. rec=[183] Log Message=Barrier completion status for nodeid 1 = 1. rec=[184] Log Message=Barrier completion status for nodeid 3 = 1. rec=[185] Log Message=Barrier completion status for nodeid 4 = 0. rec=[186] Log Message=Received ringid(192.168.0.59:48956) seq d rec=[187] Log Message=Delivering c to d rec=[188] Log Message=Delivering MCAST message with seq d to pending delivery queue rec=[189] Log Message=confchg entries 4 rec=[190] Log Message=Barrier Start Received From 4 rec=[191] Log Message=Barrier completion status for nodeid 2 = 1. rec=[192] Log Message=Barrier completion status for nodeid 1 = 1. rec=[193] Log Message=Barrier completion status for nodeid 3 = 1. rec=[194] Log Message=Barrier completion status for nodeid 4 = 1. rec=[195] Log Message=Synchronization barrier completed rec=[196] Log Message=Committing synchronization for (dummy CLM service) rec=[197] Log Message=Synchronization actions starting for (dummy AMF service) rec=[198] Log Message=releasing messages up to and including a rec=[199] Log Message=releasing messages up to and including d rec=[200] Log Message=Received ringid(192.168.0.59:48956) seq e rec=[201] Log Message=Delivering d to e rec=[202] Log Message=Delivering MCAST message with seq e to pending delivery queue rec=[203] Log Message=confchg entries 4 rec=[204] Log Message=Barrier Start Received From 4 rec=[205] Log Message=Barrier completion status for nodeid 2 = 0. rec=[206] Log Message=Barrier completion status for nodeid 1 = 0. rec=[207] Log Message=Barrier completion status for nodeid 3 = 0. rec=[208] Log Message=Barrier completion status for nodeid 4 = 1. rec=[209] Log Message=Received ringid(192.168.0.59:48956) seq f rec=[210] Log Message=Delivering e to f rec=[211] Log Message=Delivering MCAST message with seq f to pending delivery queue rec=[212] Log Message=confchg entries 4 rec=[213] Log Message=Barrier Start Received From 2 rec=[214] Log Message=Barrier completion status for nodeid 2 = 1. rec=[215] Log Message=Barrier completion status for nodeid 1 = 0. rec=[216] Log Message=Barrier completion status for nodeid 3 = 0. rec=[217] Log Message=Barrier completion status for nodeid 4 = 1. rec=[218] Log Message=mcasted message added to pending queue rec=[219] Log Message=Delivering f to 10 rec=[220] Log Message=Delivering MCAST message with seq 10 to pending delivery queue rec=[221] Log Message=confchg entries 4 rec=[222] Log Message=Barrier Start Received From 1 rec=[223] Log Message=Barrier completion status for nodeid 2 = 1. rec=[224] Log Message=Barrier completion status for nodeid 1 = 1. rec=[225] Log Message=Barrier completion status for nodeid 3 = 0. rec=[226] Log Message=Barrier completion status for nodeid 4 = 1. rec=[227] Log Message=Received ringid(192.168.0.59:48956) seq 10 rec=[228] Log Message=Received ringid(192.168.0.59:48956) seq 11 rec=[229] Log Message=Delivering 10 to 11 rec=[230] Log Message=Delivering MCAST message with seq 11 to pending delivery queue rec=[231] Log Message=confchg entries 4 rec=[232] Log Message=Barrier Start Received From 3 rec=[233] Log Message=Barrier completion status for nodeid 2 = 1. rec=[234] Log Message=Barrier completion status for nodeid 1 = 1. rec=[235] Log Message=Barrier completion status for nodeid 3 = 1. rec=[236] Log Message=Barrier completion status for nodeid 4 = 1. rec=[237] Log Message=Synchronization barrier completed rec=[238] Log Message=Committing synchronization for (dummy AMF service) rec=[239] Log Message=Synchronization actions starting for (dummy CKPT service) rec=[240] Log Message=releasing messages up to and including f rec=[241] Log Message=releasing messages up to and including 11 rec=[242] Log Message=Received ringid(192.168.0.59:48956) seq 12 rec=[243] Log Message=Delivering 11 to 12 rec=[244] Log Message=Delivering MCAST message with seq 12 to pending delivery queue rec=[245] Log Message=confchg entries 4 rec=[246] Log Message=Barrier Start Received From 3 rec=[247] Log Message=Barrier completion status for nodeid 2 = 0. rec=[248] Log Message=Barrier completion status for nodeid 1 = 0. rec=[249] Log Message=Barrier completion status for nodeid 3 = 1. rec=[250] Log Message=Barrier completion status for nodeid 4 = 0. rec=[251] Log Message=Received ringid(192.168.0.59:48956) seq 13 rec=[252] Log Message=Delivering 12 to 13 rec=[253] Log Message=Delivering MCAST message with seq 13 to pending delivery queue rec=[254] Log Message=confchg entries 4 rec=[255] Log Message=Barrier Start Received From 4 rec=[256] Log Message=Barrier completion status for nodeid 2 = 0. rec=[257] Log Message=Barrier completion status for nodeid 1 = 0. rec=[258] Log Message=Barrier completion status for nodeid 3 = 1. rec=[259] Log Message=Barrier completion status for nodeid 4 = 1. rec=[260] Log Message=Received ringid(192.168.0.59:48956) seq 14 rec=[261] Log Message=Delivering 13 to 14 rec=[262] Log Message=Delivering MCAST message with seq 14 to pending delivery queue rec=[263] Log Message=confchg entries 4 rec=[264] Log Message=Barrier Start Received From 2 rec=[265] Log Message=Barrier completion status for nodeid 2 = 1. rec=[266] Log Message=Barrier completion status for nodeid 1 = 0. rec=[267] Log Message=Barrier completion status for nodeid 3 = 1. rec=[268] Log Message=Barrier completion status for nodeid 4 = 1. rec=[269] Log Message=mcasted message added to pending queue rec=[270] Log Message=Delivering 14 to 15 rec=[271] Log Message=Delivering MCAST message with seq 15 to pending delivery queue rec=[272] Log Message=confchg entries 4 rec=[273] Log Message=Barrier Start Received From 1 rec=[274] Log Message=Barrier completion status for nodeid 2 = 1. rec=[275] Log Message=Barrier completion status for nodeid 1 = 1. rec=[276] Log Message=Barrier completion status for nodeid 3 = 1. rec=[277] Log Message=Barrier completion status for nodeid 4 = 1. rec=[278] Log Message=Synchronization barrier completed rec=[279] Log Message=Committing synchronization for (dummy CKPT service) rec=[280] Log Message=Synchronization actions starting for (dummy EVT service) rec=[281] Log Message=Received ringid(192.168.0.59:48956) seq 15 rec=[282] Log Message=mcasted message added to pending queue rec=[283] Log Message=releasing messages up to and including 14 rec=[284] Log Message=Delivering 15 to 16 rec=[285] Log Message=Delivering MCAST message with seq 16 to pending delivery queue rec=[286] Log Message=Received ringid(192.168.0.59:48956) seq 16 rec=[287] Log Message=Received ringid(192.168.0.59:48956) seq 17 rec=[288] Log Message=Delivering 16 to 17 rec=[289] Log Message=Delivering MCAST message with seq 17 to pending delivery queue rec=[290] Log Message=Received ringid(192.168.0.59:48956) seq 18 rec=[291] Log Message=Delivering 17 to 18 rec=[292] Log Message=Delivering MCAST message with seq 18 to pending delivery queue rec=[293] Log Message=Received ringid(192.168.0.59:48956) seq 19 rec=[294] Log Message=Delivering 18 to 19 rec=[295] Log Message=Delivering MCAST message with seq 19 to pending delivery queue rec=[296] Log Message=releasing messages up to and including 15 rec=[297] Log Message=Received ringid(192.168.0.59:48956) seq 1a rec=[298] Log Message=Delivering 19 to 1a rec=[299] Log Message=Delivering MCAST message with seq 1a to pending delivery queue rec=[300] Log Message=mcasted message added to pending queue rec=[301] Log Message=releasing messages up to and including 19 rec=[302] Log Message=Delivering 1a to 1b rec=[303] Log Message=Delivering MCAST message with seq 1b to pending delivery queue rec=[304] Log Message=Received ringid(192.168.0.59:48956) seq 1b rec=[305] Log Message=Received ringid(192.168.0.59:48956) seq 1c rec=[306] Log Message=Delivering 1b to 1c rec=[307] Log Message=Delivering MCAST message with seq 1c to pending delivery queue rec=[308] Log Message=Received ringid(192.168.0.59:48956) seq 1d rec=[309] Log Message=Delivering 1c to 1d rec=[310] Log Message=Delivering MCAST message with seq 1d to pending delivery queue rec=[311] Log Message=releasing messages up to and including 1a rec=[312] Log Message=releasing messages up to and including 1d rec=[313] Log Message=Received ringid(192.168.0.59:48956) seq 1e rec=[314] Log Message=Delivering 1d to 1e rec=[315] Log Message=Delivering MCAST message with seq 1e to pending delivery queue rec=[316] Log Message=confchg entries 4 rec=[317] Log Message=Barrier Start Received From 4 rec=[318] Log Message=Barrier completion status for nodeid 2 = 0. rec=[319] Log Message=Barrier completion status for nodeid 1 = 0. rec=[320] Log Message=Barrier completion status for nodeid 3 = 0. rec=[321] Log Message=Barrier completion status for nodeid 4 = 1. rec=[322] Log Message=Received ringid(192.168.0.59:48956) seq 1f rec=[323] Log Message=Delivering 1e to 1f rec=[324] Log Message=Delivering MCAST message with seq 1f to pending delivery queue rec=[325] Log Message=confchg entries 4 rec=[326] Log Message=Barrier Start Received From 2 rec=[327] Log Message=Barrier completion status for nodeid 2 = 1. rec=[328] Log Message=Barrier completion status for nodeid 1 = 0. rec=[329] Log Message=Barrier completion status for nodeid 3 = 0. rec=[330] Log Message=Barrier completion status for nodeid 4 = 1. rec=[331] Log Message=mcasted message added to pending queue rec=[332] Log Message=Delivering 1f to 20 rec=[333] Log Message=Delivering MCAST message with seq 20 to pending delivery queue rec=[334] Log Message=confchg entries 4 rec=[335] Log Message=Barrier Start Received From 1 rec=[336] Log Message=Barrier completion status for nodeid 2 = 1. rec=[337] Log Message=Barrier completion status for nodeid 1 = 1. rec=[338] Log Message=Barrier completion status for nodeid 3 = 0. rec=[339] Log Message=Barrier completion status for nodeid 4 = 1. rec=[340] Log Message=Received ringid(192.168.0.59:48956) seq 20 rec=[341] Log Message=Received ringid(192.168.0.59:48956) seq 21 rec=[342] Log Message=Delivering 20 to 21 rec=[343] Log Message=Delivering MCAST message with seq 21 to pending delivery queue rec=[344] Log Message=confchg entries 4 rec=[345] Log Message=Barrier Start Received From 3 rec=[346] Log Message=Barrier completion status for nodeid 2 = 1. rec=[347] Log Message=Barrier completion status for nodeid 1 = 1. rec=[348] Log Message=Barrier completion status for nodeid 3 = 1. rec=[349] Log Message=Barrier completion status for nodeid 4 = 1. rec=[350] Log Message=Synchronization barrier completed rec=[351] Log Message=Committing synchronization for (dummy EVT service) rec=[352] Log Message=Synchronization actions starting for (corosync cluster closed process group service v1.01) rec=[353] Log Message=releasing messages up to and including 1f rec=[354] Log Message=mcasted message added to pending queue rec=[355] Log Message=Received ringid(192.168.0.59:48956) seq 22 rec=[356] Log Message=Delivering 21 to 22 rec=[357] Log Message=Delivering MCAST message with seq 22 to pending delivery queue rec=[358] Log Message=Received ringid(192.168.0.59:48956) seq 23 rec=[359] Log Message=Delivering 22 to 23 rec=[360] Log Message=Delivering MCAST message with seq 23 to pending delivery queue rec=[361] Log Message=got joinlist message from node 3 rec=[362] Log Message=Received ringid(192.168.0.59:48956) seq 24 rec=[363] Log Message=Delivering 23 to 24 rec=[364] Log Message=Delivering MCAST message with seq 24 to pending delivery queue rec=[365] Log Message=Received ringid(192.168.0.59:48956) seq 25 rec=[366] Log Message=Delivering 24 to 25 rec=[367] Log Message=Delivering MCAST message with seq 25 to pending delivery queue rec=[368] Log Message=got joinlist message from node 4 rec=[369] Log Message=Received ringid(192.168.0.59:48956) seq 26 rec=[370] Log Message=Delivering 25 to 26 rec=[371] Log Message=Delivering MCAST message with seq 26 to pending delivery queue rec=[372] Log Message=Received ringid(192.168.0.59:48956) seq 27 rec=[373] Log Message=Delivering 26 to 27 rec=[374] Log Message=Delivering MCAST message with seq 27 to pending delivery queue rec=[375] Log Message=got joinlist message from node 2 rec=[376] Log Message=mcasted message added to pending queue rec=[377] Log Message=releasing messages up to and including 21 rec=[378] Log Message=Delivering 27 to 29 rec=[379] Log Message=Delivering MCAST message with seq 28 to pending delivery queue rec=[380] Log Message=Delivering MCAST message with seq 29 to pending delivery queue rec=[381] Log Message=comparing: sender r(0) ip(192.168.0.63) ; members(old:0 left:0) rec=[382] Log Message=comparing: sender r(0) ip(192.168.0.59) ; members(old:3 left:0) rec=[383] Log Message=comparing: sender r(0) ip(192.168.0.70) ; members(old:3 left:0) rec=[384] Log Message=comparing: sender r(0) ip(192.168.0.68) ; members(old:3 left:0) rec=[385] Log Message=chosen downlist: sender r(0) ip(192.168.0.59) ; members(old:3 left:0) rec=[386] Log Message=Received ringid(192.168.0.59:48956) seq 28 rec=[387] Log Message=Received ringid(192.168.0.59:48956) seq 29 rec=[388] Log Message=Received ringid(192.168.0.59:48956) seq 2a rec=[389] Log Message=Delivering 29 to 2a rec=[390] Log Message=Delivering MCAST message with seq 2a to pending delivery queue rec=[391] Log Message=confchg entries 4 rec=[392] Log Message=Barrier Start Received From 3 rec=[393] Log Message=Barrier completion status for nodeid 2 = 0. rec=[394] Log Message=Barrier completion status for nodeid 1 = 0. rec=[395] Log Message=Barrier completion status for nodeid 3 = 1. rec=[396] Log Message=Barrier completion status for nodeid 4 = 0. rec=[397] Log Message=Received ringid(192.168.0.59:48956) seq 2b rec=[398] Log Message=Delivering 2a to 2b rec=[399] Log Message=Delivering MCAST message with seq 2b to pending delivery queue rec=[400] Log Message=confchg entries 4 rec=[401] Log Message=Barrier Start Received From 4 rec=[402] Log Message=Barrier completion status for nodeid 2 = 0. rec=[403] Log Message=Barrier completion status for nodeid 1 = 0. rec=[404] Log Message=Barrier completion status for nodeid 3 = 1. rec=[405] Log Message=Barrier completion status for nodeid 4 = 1. rec=[406] Log Message=Received ringid(192.168.0.59:48956) seq 2c rec=[407] Log Message=Delivering 2b to 2c rec=[408] Log Message=Delivering MCAST message with seq 2c to pending delivery queue rec=[409] Log Message=confchg entries 4 rec=[410] Log Message=Barrier Start Received From 2 rec=[411] Log Message=Barrier completion status for nodeid 2 = 1. rec=[412] Log Message=Barrier completion status for nodeid 1 = 0. rec=[413] Log Message=Barrier completion status for nodeid 3 = 1. rec=[414] Log Message=Barrier completion status for nodeid 4 = 1. rec=[415] Log Message=mcasted message added to pending queue rec=[416] Log Message=releasing messages up to and including 27 rec=[417] Log Message=Delivering 2c to 2d rec=[418] Log Message=Delivering MCAST message with seq 2d to pending delivery queue rec=[419] Log Message=confchg entries 4 rec=[420] Log Message=Barrier Start Received From 1 rec=[421] Log Message=Barrier completion status for nodeid 2 = 1. rec=[422] Log Message=Barrier completion status for nodeid 1 = 1. rec=[423] Log Message=Barrier completion status for nodeid 3 = 1. rec=[424] Log Message=Barrier completion status for nodeid 4 = 1. rec=[425] Log Message=Synchronization barrier completed rec=[426] Log Message=Committing synchronization for (corosync cluster closed process group service v1.01) rec=[427] Log Message=Received ringid(192.168.0.59:48956) seq 2d rec=[428] Log Message=Received ringid(192.168.0.59:48956) seq 2e rec=[429] Log Message=Delivering 2d to 2e rec=[430] Log Message=Delivering MCAST message with seq 2e to pending delivery queue rec=[431] Log Message=Received ringid(192.168.0.59:48956) seq 2f rec=[432] Log Message=Delivering 2e to 2f rec=[433] Log Message=Delivering MCAST message with seq 2f to pending delivery queue rec=[434] Log Message=Received ringid(192.168.0.59:48956) seq 30 rec=[435] Log Message=Delivering 2f to 30 rec=[436] Log Message=Delivering MCAST message with seq 30 to pending delivery queue rec=[437] Log Message=mcasted message added to pending queue rec=[438] Log Message=releasing messages up to and including 2c rec=[439] Log Message=Delivering 30 to 31 rec=[440] Log Message=Delivering MCAST message with seq 31 to pending delivery queue rec=[441] Log Message=Completed service synchronization, ready to provide service. rec=[442] Log Message=Received ringid(192.168.0.59:48956) seq 31 rec=[443] Log Message=releasing messages up to and including 30 rec=[444] Log Message=releasing messages up to and including 31 rec=[445] Log Message=lib_init_fn: conn=0x17aee00, cpd=0x17ae1b0 rec=[446] Log Message=mcasted message added to pending queue rec=[447] Log Message=Delivering 31 to 32 rec=[448] Log Message=Delivering MCAST message with seq 32 to pending delivery queue rec=[449] Log Message=got procjoin message from cluster node 1 rec=[450] Log Message=Received ringid(192.168.0.59:48956) seq 32 rec=[451] Log Message=releasing messages up to and including 32 rec=[452] Log Message=exit_fn for conn=0x17aee00 rec=[453] Log Message=mcasted message added to pending queue rec=[454] Log Message=Delivering 32 to 33 rec=[455] Log Message=Delivering MCAST message with seq 33 to pending delivery queue rec=[456] Log Message=got procleave message from cluster node 1 rec=[457] Log Message=Received ringid(192.168.0.59:48956) seq 33 rec=[458] Log Message=releasing messages up to and including 33 rec=[459] Log Message=lib_init_fn: conn=0x17aee00, cpd=0x17ae1b0 rec=[460] Log Message=mcasted message added to pending queue rec=[461] Log Message=Delivering 33 to 34 rec=[462] Log Message=Delivering MCAST message with seq 34 to pending delivery queue rec=[463] Log Message=got procjoin message from cluster node 1 rec=[464] Log Message=Received ringid(192.168.0.59:48956) seq 34 rec=[465] Log Message=releasing messages up to and including 34 rec=[466] Log Message=exit_fn for conn=0x17aee00 rec=[467] Log Message=mcasted message added to pending queue rec=[468] Log Message=Delivering 34 to 35 rec=[469] Log Message=Delivering MCAST message with seq 35 to pending delivery queue rec=[470] Log Message=got procleave message from cluster node 1 rec=[471] Log Message=Received ringid(192.168.0.59:48956) seq 35 rec=[472] Log Message=releasing messages up to and including 35 rec=[473] Log Message=lib_init_fn: conn=0x17aee00, cpd=0x17ae1b0 rec=[474] Log Message=mcasted message added to pending queue rec=[475] Log Message=Delivering 35 to 36 rec=[476] Log Message=Delivering MCAST message with seq 36 to pending delivery queue rec=[477] Log Message=got procjoin message from cluster node 1 rec=[478] Log Message=Received ringid(192.168.0.59:48956) seq 36 rec=[479] Log Message=releasing messages up to and including 36 rec=[480] Log Message=lib_init_fn: conn=0x17b3160, cpd=0x17b74c0 rec=[481] Log Message=mcasted message added to pending queue rec=[482] Log Message=Delivering 36 to 37 rec=[483] Log Message=Delivering MCAST message with seq 37 to pending delivery queue rec=[484] Log Message=got procjoin message from cluster node 1 rec=[485] Log Message=Received ringid(192.168.0.59:48956) seq 37 rec=[486] Log Message=got mcast request on 0x17b3160 rec=[487] Log Message=mcasted message added to pending queue rec=[488] Log Message=Delivering 37 to 38 rec=[489] Log Message=Delivering MCAST message with seq 38 to pending delivery queue rec=[490] Log Message=Received ringid(192.168.0.59:48956) seq 38 rec=[491] Log Message=Received ringid(192.168.0.59:48956) seq 39 rec=[492] Log Message=Delivering 38 to 39 rec=[493] Log Message=Delivering MCAST message with seq 39 to pending delivery queue rec=[494] Log Message=releasing messages up to and including 37 rec=[495] Log Message=releasing messages up to and including 39 rec=[496] Log Message=Received ringid(192.168.0.59:48956) seq 3a rec=[497] Log Message=Delivering 39 to 3a rec=[498] Log Message=Delivering MCAST message with seq 3a to pending delivery queue rec=[499] Log Message=releasing messages up to and including 3a rec=[500] Log Message=Received ringid(192.168.0.59:48956) seq 3b rec=[501] Log Message=Delivering 3a to 3b rec=[502] Log Message=Delivering MCAST message with seq 3b to pending delivery queue rec=[503] Log Message=releasing messages up to and including 3b rec=[504] Log Message=entering GATHER state from 11. rec=[505] Log Message=entering GATHER state from 11. rec=[506] Log Message=entering GATHER state from 11. rec=[507] Log Message=Creating commit token because I am the rep. rec=[508] Log Message=Saving state aru 3b high seq received 3b rec=[509] Log Message=Storing new sequence id for ring bf40 rec=[510] Log Message=entering COMMIT state. rec=[511] Log Message=got commit token rec=[512] Log Message=entering RECOVERY state. rec=[513] Log Message=TRANS [0] member 192.168.0.63: rec=[514] Log Message=position [0] member 192.168.0.63: rec=[515] Log Message=previous ring seq bf3c rep 192.168.0.59 rec=[516] Log Message=aru 3b high delivered 3b received flag 1 rec=[517] Log Message=Did not need to originate any messages in recovery. rec=[518] Log Message=got commit token rec=[519] Log Message=Sending initial ORF token rec=[520] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0 rec=[521] Log Message=install seq 0 aru 0 high seq received 0 rec=[522] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 rec=[523] Log Message=install seq 0 aru 0 high seq received 0 rec=[524] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 rec=[525] Log Message=install seq 0 aru 0 high seq received 0 rec=[526] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 rec=[527] Log Message=install seq 0 aru 0 high seq received 0 rec=[528] Log Message=retrans flag count 4 token aru 0 install seq 0 aru 0 0 rec=[529] Log Message=Resetting old ring state rec=[530] Log Message=recovery to regular 1-0 rec=[531] Log Message=Delivering to app 3c to 3b rec=[532] Log Message=This node is within the primary component and will provide service. rec=[533] Log Message=entering OPERATIONAL state. rec=[534] Log Message=A processor joined or left the membership and a new membership was formed. rec=[535] Log Message=mcasted message added to pending queue rec=[536] Log Message=Delivering 0 to 1 rec=[537] Log Message=Delivering MCAST message with seq 1 to pending delivery queue rec=[538] Log Message=confchg entries 1 rec=[539] Log Message=Barrier Start Received From 1 rec=[540] Log Message=Barrier completion status for nodeid 1 = 1. rec=[541] Log Message=Synchronization barrier completed rec=[542] Log Message=Synchronization actions starting for (dummy CLM service) rec=[543] Log Message=mcasted message added to pending queue rec=[544] Log Message=Delivering 1 to 2 rec=[545] Log Message=Delivering MCAST message with seq 2 to pending delivery queue rec=[546] Log Message=mcasted message added to pending queue rec=[547] Log Message=releasing messages up to and including 1 rec=[548] Log Message=Delivering 2 to 3 rec=[549] Log Message=Delivering MCAST message with seq 3 to pending delivery queue rec=[550] Log Message=confchg entries 1 rec=[551] Log Message=Barrier Start Received From 1 rec=[552] Log Message=Barrier completion status for nodeid 1 = 1. rec=[553] Log Message=Synchronization barrier completed rec=[554] Log Message=Committing synchronization for (dummy CLM service) rec=[555] Log Message=Synchronization actions starting for (dummy AMF service) rec=[556] Log Message=releasing messages up to and including 2 rec=[557] Log Message=mcasted message added to pending queue rec=[558] Log Message=releasing messages up to and including 3 rec=[559] Log Message=Delivering 3 to 4 rec=[560] Log Message=Delivering MCAST message with seq 4 to pending delivery queue rec=[561] Log Message=confchg entries 1 rec=[562] Log Message=Barrier Start Received From 1 rec=[563] Log Message=Barrier completion status for nodeid 1 = 1. rec=[564] Log Message=Synchronization barrier completed rec=[565] Log Message=Committing synchronization for (dummy AMF service) rec=[566] Log Message=Synchronization actions starting for (dummy CKPT service) rec=[567] Log Message=mcasted message added to pending queue rec=[568] Log Message=releasing messages up to and including 4 rec=[569] Log Message=Delivering 4 to 5 rec=[570] Log Message=Delivering MCAST message with seq 5 to pending delivery queue rec=[571] Log Message=confchg entries 1 rec=[572] Log Message=Barrier Start Received From 1 rec=[573] Log Message=Barrier completion status for nodeid 1 = 1. rec=[574] Log Message=Synchronization barrier completed rec=[575] Log Message=Committing synchronization for (dummy CKPT service) rec=[576] Log Message=Synchronization actions starting for (dummy EVT service) rec=[577] Log Message=mcasted message added to pending queue rec=[578] Log Message=Delivering 5 to 6 rec=[579] Log Message=Delivering MCAST message with seq 6 to pending delivery queue rec=[580] Log Message=mcasted message added to pending queue rec=[581] Log Message=releasing messages up to and including 5 rec=[582] Log Message=Delivering 6 to 7 rec=[583] Log Message=Delivering MCAST message with seq 7 to pending delivery queue rec=[584] Log Message=releasing messages up to and including 6 rec=[585] Log Message=mcasted message added to pending queue rec=[586] Log Message=releasing messages up to and including 7 rec=[587] Log Message=Delivering 7 to 8 rec=[588] Log Message=Delivering MCAST message with seq 8 to pending delivery queue rec=[589] Log Message=confchg entries 1 rec=[590] Log Message=Barrier Start Received From 1 rec=[591] Log Message=Barrier completion status for nodeid 1 = 1. rec=[592] Log Message=Synchronization barrier completed rec=[593] Log Message=Committing synchronization for (dummy EVT service) rec=[594] Log Message=Synchronization actions starting for (corosync cluster closed process group service v1.01) rec=[595] Log Message=mcasted message added to pending queue rec=[596] Log Message=mcasted message added to pending queue rec=[597] Log Message=Delivering 8 to a rec=[598] Log Message=Delivering MCAST message with seq 9 to pending delivery queue rec=[599] Log Message=Delivering MCAST message with seq a to pending delivery queue rec=[600] Log Message=comparing: sender r(0) ip(192.168.0.63) ; members(old:4 left:3) rec=[601] Log Message=chosen downlist: sender r(0) ip(192.168.0.63) ; members(old:4 left:3) rec=[602] Log Message=got joinlist message from node 1 rec=[603] Log Message=mcasted message added to pending queue rec=[604] Log Message=releasing messages up to and including 8 rec=[605] Log Message=Delivering a to b rec=[606] Log Message=Delivering MCAST message with seq b to pending delivery queue rec=[607] Log Message=confchg entries 1 rec=[608] Log Message=Barrier Start Received From 1 rec=[609] Log Message=Barrier completion status for nodeid 1 = 1. rec=[610] Log Message=Synchronization barrier completed rec=[611] Log Message=Committing synchronization for (corosync cluster closed process group service v1.01) rec=[612] Log Message=mcasted message added to pending queue rec=[613] Log Message=releasing messages up to and including a rec=[614] Log Message=Delivering b to c rec=[615] Log Message=Delivering MCAST message with seq c to pending delivery queue rec=[616] Log Message=Completed service synchronization, ready to provide service. rec=[617] Log Message=releasing messages up to and including b rec=[618] Log Message=releasing messages up to and including c rec=[619] Log Message=entering GATHER state from 7. rec=[620] Log Message=entering GATHER state from 11. rec=[621] Log Message=got commit token rec=[622] Log Message=Saving state aru c high seq received c rec=[623] Log Message=Storing new sequence id for ring bf44 rec=[624] Log Message=entering COMMIT state. rec=[625] Log Message=got commit token rec=[626] Log Message=entering RECOVERY state. rec=[627] Log Message=TRANS [0] member 192.168.0.63: rec=[628] Log Message=position [0] member 192.168.0.59: rec=[629] Log Message=previous ring seq bf40 rep 192.168.0.59 rec=[630] Log Message=aru 2 high delivered 1 received flag 1 rec=[631] Log Message=position [1] member 192.168.0.63: rec=[632] Log Message=previous ring seq bf40 rep 192.168.0.63 rec=[633] Log Message=aru c high delivered c received flag 1 rec=[634] Log Message=position [2] member 192.168.0.68: rec=[635] Log Message=previous ring seq bf40 rep 192.168.0.59 rec=[636] Log Message=aru 2 high delivered 1 received flag 1 rec=[637] Log Message=position [3] member 192.168.0.70: rec=[638] Log Message=previous ring seq bf40 rep 192.168.0.59 rec=[639] Log Message=aru 2 high delivered 2 received flag 1 rec=[640] Log Message=Did not need to originate any messages in recovery. rec=[641] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru ffffffff rec=[642] Log Message=install seq 0 aru 0 high seq received 0 rec=[643] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 rec=[644] Log Message=install seq 0 aru 0 high seq received 0 rec=[645] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 rec=[646] Log Message=install seq 0 aru 0 high seq received 0 rec=[647] Log Message=token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 rec=[648] Log Message=install seq 0 aru 0 high seq received 0 rec=[649] Log Message=retrans flag count 4 token aru 0 install seq 0 aru 0 0 rec=[650] Log Message=Resetting old ring state rec=[651] Log Message=recovery to regular 1-0 rec=[652] Log Message=Delivering to app d to c rec=[653] Log Message=This node is within the primary component and will provide service. rec=[654] Log Message=entering OPERATIONAL state. rec=[655] Log Message=A processor joined or left the membership and a new membership was formed. rec=[656] Log Message=mcasted message added to pending queue rec=[657] Log Message=Delivering 0 to 1 rec=[658] Log Message=Delivering MCAST message with seq 1 to pending delivery queue rec=[659] Log Message=confchg entries 4 rec=[660] Log Message=Barrier Start Received From 1 rec=[661] Log Message=Barrier completion status for nodeid 2 = 0. rec=[662] Log Message=Barrier completion status for nodeid 1 = 1. rec=[663] Log Message=Barrier completion status for nodeid 3 = 0. rec=[664] Log Message=Barrier completion status for nodeid 4 = 0. rec=[665] Log Message=Received ringid(192.168.0.59:48964) seq 1 rec=[666] Log Message=Received ringid(192.168.0.59:48964) seq 2 rec=[667] Log Message=Delivering 1 to 2 rec=[668] Log Message=Delivering MCAST message with seq 2 to pending delivery queue rec=[669] Log Message=confchg entries 4 rec=[670] Log Message=Barrier Start Received From 3 rec=[671] Log Message=Barrier completion status for nodeid 2 = 0. rec=[672] Log Message=Barrier completion status for nodeid 1 = 1. rec=[673] Log Message=Barrier completion status for nodeid 3 = 1. rec=[674] Log Message=Barrier completion status for nodeid 4 = 0. rec=[675] Log Message=Received ringid(192.168.0.59:48964) seq 3 rec=[676] Log Message=Delivering 2 to 3 rec=[677] Log Message=Delivering MCAST message with seq 3 to pending delivery queue rec=[678] Log Message=confchg entries 4 rec=[679] Log Message=Barrier Start Received From 4 rec=[680] Log Message=Barrier completion status for nodeid 2 = 0. rec=[681] Log Message=Barrier completion status for nodeid 1 = 1. rec=[682] Log Message=Barrier completion status for nodeid 3 = 1. rec=[683] Log Message=Barrier completion status for nodeid 4 = 1. rec=[684] Log Message=Received ringid(192.168.0.59:48964) seq 4 rec=[685] Log Message=Delivering 3 to 4 rec=[686] Log Message=Delivering MCAST message with seq 4 to pending delivery queue rec=[687] Log Message=confchg entries 4 rec=[688] Log Message=Received ringid(192.168.0.59:48964) seq 5 rec=[689] Log Message=Delivering 4 to 5 rec=[690] Log Message=Delivering MCAST message with seq 5 to pending delivery queue rec=[691] Log Message=confchg entries 4 rec=[692] Log Message=Barrier Start Received From 2 rec=[693] Log Message=Barrier completion status for nodeid 2 = 1. rec=[694] Log Message=Barrier completion status for nodeid 1 = 1. rec=[695] Log Message=Barrier completion status for nodeid 3 = 1. rec=[696] Log Message=Barrier completion status for nodeid 4 = 1. rec=[697] Log Message=Synchronization barrier completed rec=[698] Log Message=Synchronization actions starting for (dummy CLM service) rec=[699] Log Message=Received ringid(192.168.0.59:48964) seq 6 rec=[700] Log Message=Delivering 5 to 6 rec=[701] Log Message=Delivering MCAST message with seq 6 to pending delivery queue rec=[702] Log Message=mcasted message added to pending queue rec=[703] Log Message=releasing messages up to and including 5 rec=[704] Log Message=Delivering 6 to 7 rec=[705] Log Message=Delivering MCAST message with seq 7 to pending delivery queue rec=[706] Log Message=Received ringid(192.168.0.59:48964) seq 7 rec=[707] Log Message=Received ringid(192.168.0.59:48964) seq 8 rec=[708] Log Message=Delivering 7 to 8 rec=[709] Log Message=Delivering MCAST message with seq 8 to pending delivery queue rec=[710] Log Message=Received ringid(192.168.0.59:48964) seq 9 rec=[711] Log Message=Delivering 8 to 9 rec=[712] Log Message=Delivering MCAST message with seq 9 to pending delivery queue rec=[713] Log Message=Received ringid(192.168.0.59:48964) seq a rec=[714] Log Message=Delivering 9 to a rec=[715] Log Message=Delivering MCAST message with seq a to pending delivery queue rec=[716] Log Message=confchg entries 4 rec=[717] Log Message=Barrier Start Received From 2 rec=[718] Log Message=Barrier completion status for nodeid 2 = 1. rec=[719] Log Message=Barrier completion status for nodeid 1 = 0. rec=[720] Log Message=Barrier completion status for nodeid 3 = 0. rec=[721] Log Message=Barrier completion status for nodeid 4 = 0. rec=[722] Log Message=mcasted message added to pending queue rec=[723] Log Message=releasing messages up to and including 6 rec=[724] Log Message=Delivering a to b rec=[725] Log Message=Delivering MCAST message with seq b to pending delivery queue rec=[726] Log Message=confchg entries 4 rec=[727] Log Message=Barrier Start Received From 1 rec=[728] Log Message=Barrier completion status for nodeid 2 = 1. rec=[729] Log Message=Barrier completion status for nodeid 1 = 1. rec=[730] Log Message=Barrier completion status for nodeid 3 = 0. rec=[731] Log Message=Barrier completion status for nodeid 4 = 0. rec=[732] Log Message=Received ringid(192.168.0.59:48964) seq b rec=[733] Log Message=Received ringid(192.168.0.59:48964) seq c rec=[734] Log Message=Delivering b to c rec=[735] Log Message=Delivering MCAST message with seq c to pending delivery queue rec=[736] Log Message=confchg entries 4 rec=[737] Log Message=Barrier Start Received From 3 rec=[738] Log Message=Barrier completion status for nodeid 2 = 1. rec=[739] Log Message=Barrier completion status for nodeid 1 = 1. rec=[740] Log Message=Barrier completion status for nodeid 3 = 1. rec=[741] Log Message=Barrier completion status for nodeid 4 = 0. rec=[742] Log Message=Received ringid(192.168.0.59:48964) seq d rec=[743] Log Message=Delivering c to d rec=[744] Log Message=Delivering MCAST message with seq d to pending delivery queue rec=[745] Log Message=confchg entries 4 rec=[746] Log Message=Barrier Start Received From 4 rec=[747] Log Message=Barrier completion status for nodeid 2 = 1. rec=[748] Log Message=Barrier completion status for nodeid 1 = 1. rec=[749] Log Message=Barrier completion status for nodeid 3 = 1. rec=[750] Log Message=Barrier completion status for nodeid 4 = 1. rec=[751] Log Message=Synchronization barrier completed rec=[752] Log Message=Committing synchronization for (dummy CLM service) rec=[753] Log Message=Synchronization actions starting for (dummy AMF service) rec=[754] Log Message=releasing messages up to and including a rec=[755] Log Message=releasing messages up to and including d rec=[756] Log Message=Received ringid(192.168.0.59:48964) seq e rec=[757] Log Message=Delivering d to e rec=[758] Log Message=Delivering MCAST message with seq e to pending delivery queue rec=[759] Log Message=confchg entries 4 rec=[760] Log Message=Barrier Start Received From 4 rec=[761] Log Message=Barrier completion status for nodeid 2 = 0. rec=[762] Log Message=Barrier completion status for nodeid 1 = 0. rec=[763] Log Message=Barrier completion status for nodeid 3 = 0. rec=[764] Log Message=Barrier completion status for nodeid 4 = 1. rec=[765] Log Message=Received ringid(192.168.0.59:48964) seq f rec=[766] Log Message=Delivering e to f rec=[767] Log Message=Delivering MCAST message with seq f to pending delivery queue rec=[768] Log Message=confchg entries 4 rec=[769] Log Message=Barrier Start Received From 2 rec=[770] Log Message=Barrier completion status for nodeid 2 = 1. rec=[771] Log Message=Barrier completion status for nodeid 1 = 0. rec=[772] Log Message=Barrier completion status for nodeid 3 = 0. rec=[773] Log Message=Barrier completion status for nodeid 4 = 1. rec=[774] Log Message=mcasted message added to pending queue rec=[775] Log Message=Delivering f to 10 rec=[776] Log Message=Delivering MCAST message with seq 10 to pending delivery queue rec=[777] Log Message=confchg entries 4 rec=[778] Log Message=Barrier Start Received From 1 rec=[779] Log Message=Barrier completion status for nodeid 2 = 1. rec=[780] Log Message=Barrier completion status for nodeid 1 = 1. rec=[781] Log Message=Barrier completion status for nodeid 3 = 0. rec=[782] Log Message=Barrier completion status for nodeid 4 = 1. rec=[783] Log Message=Received ringid(192.168.0.59:48964) seq 10 rec=[784] Log Message=Received ringid(192.168.0.59:48964) seq 11 rec=[785] Log Message=Delivering 10 to 11 rec=[786] Log Message=Delivering MCAST message with seq 11 to pending delivery queue rec=[787] Log Message=confchg entries 4 rec=[788] Log Message=Barrier Start Received From 3 rec=[789] Log Message=Barrier completion status for nodeid 2 = 1. rec=[790] Log Message=Barrier completion status for nodeid 1 = 1. rec=[791] Log Message=Barrier completion status for nodeid 3 = 1. rec=[792] Log Message=Barrier completion status for nodeid 4 = 1. rec=[793] Log Message=Synchronization barrier completed rec=[794] Log Message=Committing synchronization for (dummy AMF service) rec=[795] Log Message=Synchronization actions starting for (dummy CKPT service) rec=[796] Log Message=releasing messages up to and including f rec=[797] Log Message=releasing messages up to and including 11 rec=[798] Log Message=Received ringid(192.168.0.59:48964) seq 12 rec=[799] Log Message=Delivering 11 to 12 rec=[800] Log Message=Delivering MCAST message with seq 12 to pending delivery queue rec=[801] Log Message=confchg entries 4 rec=[802] Log Message=Barrier Start Received From 3 rec=[803] Log Message=Barrier completion status for nodeid 2 = 0. rec=[804] Log Message=Barrier completion status for nodeid 1 = 0. rec=[805] Log Message=Barrier completion status for nodeid 3 = 1. rec=[806] Log Message=Barrier completion status for nodeid 4 = 0. rec=[807] Log Message=Received ringid(192.168.0.59:48964) seq 13 rec=[808] Log Message=Delivering 12 to 13 rec=[809] Log Message=Delivering MCAST message with seq 13 to pending delivery queue rec=[810] Log Message=confchg entries 4 rec=[811] Log Message=Barrier Start Received From 4 rec=[812] Log Message=Barrier completion status for nodeid 2 = 0. rec=[813] Log Message=Barrier completion status for nodeid 1 = 0. rec=[814] Log Message=Barrier completion status for nodeid 3 = 1. rec=[815] Log Message=Barrier completion status for nodeid 4 = 1. rec=[816] Log Message=Received ringid(192.168.0.59:48964) seq 14 rec=[817] Log Message=Delivering 13 to 14 rec=[818] Log Message=Delivering MCAST message with seq 14 to pending delivery queue rec=[819] Log Message=confchg entries 4 rec=[820] Log Message=Barrier Start Received From 2 rec=[821] Log Message=Barrier completion status for nodeid 2 = 1. rec=[822] Log Message=Barrier completion status for nodeid 1 = 0. rec=[823] Log Message=Barrier completion status for nodeid 3 = 1. rec=[824] Log Message=Barrier completion status for nodeid 4 = 1. rec=[825] Log Message=mcasted message added to pending queue rec=[826] Log Message=Delivering 14 to 15 rec=[827] Log Message=Delivering MCAST message with seq 15 to pending delivery queue rec=[828] Log Message=confchg entries 4 rec=[829] Log Message=Barrier Start Received From 1 rec=[830] Log Message=Barrier completion status for nodeid 2 = 1. rec=[831] Log Message=Barrier completion status for nodeid 1 = 1. rec=[832] Log Message=Barrier completion status for nodeid 3 = 1. rec=[833] Log Message=Barrier completion status for nodeid 4 = 1. rec=[834] Log Message=Synchronization barrier completed rec=[835] Log Message=Committing synchronization for (dummy CKPT service) rec=[836] Log Message=Synchronization actions starting for (dummy EVT service) rec=[837] Log Message=Received ringid(192.168.0.59:48964) seq 15 rec=[838] Log Message=mcasted message added to pending queue rec=[839] Log Message=releasing messages up to and including 14 rec=[840] Log Message=Delivering 15 to 16 rec=[841] Log Message=Delivering MCAST message with seq 16 to pending delivery queue rec=[842] Log Message=Received ringid(192.168.0.59:48964) seq 16 rec=[843] Log Message=Received ringid(192.168.0.59:48964) seq 17 rec=[844] Log Message=Delivering 16 to 17 rec=[845] Log Message=Delivering MCAST message with seq 17 to pending delivery queue rec=[846] Log Message=Received ringid(192.168.0.59:48964) seq 18 rec=[847] Log Message=Delivering 17 to 18 rec=[848] Log Message=Delivering MCAST message with seq 18 to pending delivery queue rec=[849] Log Message=Received ringid(192.168.0.59:48964) seq 19 rec=[850] Log Message=Delivering 18 to 19 rec=[851] Log Message=Delivering MCAST message with seq 19 to pending delivery queue rec=[852] Log Message=releasing messages up to and including 15 rec=[853] Log Message=Received ringid(192.168.0.59:48964) seq 1a rec=[854] Log Message=Delivering 19 to 1a rec=[855] Log Message=Delivering MCAST message with seq 1a to pending delivery queue rec=[856] Log Message=mcasted message added to pending queue rec=[857] Log Message=releasing messages up to and including 19 rec=[858] Log Message=Delivering 1a to 1b rec=[859] Log Message=Delivering MCAST message with seq 1b to pending delivery queue rec=[860] Log Message=Received ringid(192.168.0.59:48964) seq 1b rec=[861] Log Message=Received ringid(192.168.0.59:48964) seq 1c rec=[862] Log Message=Delivering 1b to 1c rec=[863] Log Message=Delivering MCAST message with seq 1c to pending delivery queue rec=[864] Log Message=Received ringid(192.168.0.59:48964) seq 1d rec=[865] Log Message=Delivering 1c to 1d rec=[866] Log Message=Delivering MCAST message with seq 1d to pending delivery queue rec=[867] Log Message=releasing messages up to and including 1a rec=[868] Log Message=releasing messages up to and including 1d rec=[869] Log Message=Received ringid(192.168.0.59:48964) seq 1e rec=[870] Log Message=Delivering 1d to 1e rec=[871] Log Message=Delivering MCAST message with seq 1e to pending delivery queue rec=[872] Log Message=confchg entries 4 rec=[873] Log Message=Barrier Start Received From 4 rec=[874] Log Message=Barrier completion status for nodeid 2 = 0. rec=[875] Log Message=Barrier completion status for nodeid 1 = 0. rec=[876] Log Message=Barrier completion status for nodeid 3 = 0. rec=[877] Log Message=Barrier completion status for nodeid 4 = 1. rec=[878] Log Message=Received ringid(192.168.0.59:48964) seq 1f rec=[879] Log Message=Delivering 1e to 1f rec=[880] Log Message=Delivering MCAST message with seq 1f to pending delivery queue rec=[881] Log Message=confchg entries 4 rec=[882] Log Message=Barrier Start Received From 2 rec=[883] Log Message=Barrier completion status for nodeid 2 = 1. rec=[884] Log Message=Barrier completion status for nodeid 1 = 0. rec=[885] Log Message=Barrier completion status for nodeid 3 = 0. rec=[886] Log Message=Barrier completion status for nodeid 4 = 1. rec=[887] Log Message=mcasted message added to pending queue rec=[888] Log Message=Delivering 1f to 20 rec=[889] Log Message=Delivering MCAST message with seq 20 to pending delivery queue rec=[890] Log Message=confchg entries 4 rec=[891] Log Message=Barrier Start Received From 1 rec=[892] Log Message=Barrier completion status for nodeid 2 = 1. rec=[893] Log Message=Barrier completion status for nodeid 1 = 1. rec=[894] Log Message=Barrier completion status for nodeid 3 = 0. rec=[895] Log Message=Barrier completion status for nodeid 4 = 1. rec=[896] Log Message=Received ringid(192.168.0.59:48964) seq 20 rec=[897] Log Message=Received ringid(192.168.0.59:48964) seq 21 rec=[898] Log Message=Delivering 20 to 21 rec=[899] Log Message=Delivering MCAST message with seq 21 to pending delivery queue rec=[900] Log Message=confchg entries 4 rec=[901] Log Message=Barrier Start Received From 3 rec=[902] Log Message=Barrier completion status for nodeid 2 = 1. rec=[903] Log Message=Barrier completion status for nodeid 1 = 1. rec=[904] Log Message=Barrier completion status for nodeid 3 = 1. rec=[905] Log Message=Barrier completion status for nodeid 4 = 1. rec=[906] Log Message=Synchronization barrier completed rec=[907] Log Message=Committing synchronization for (dummy EVT service) rec=[908] Log Message=Synchronization actions starting for (corosync cluster closed process group service v1.01) rec=[909] Log Message=releasing messages up to and including 1f rec=[910] Log Message=mcasted message added to pending queue rec=[911] Log Message=Received ringid(192.168.0.59:48964) seq 22 rec=[912] Log Message=Delivering 21 to 22 rec=[913] Log Message=Delivering MCAST message with seq 22 to pending delivery queue rec=[914] Log Message=Received ringid(192.168.0.59:48964) seq 23 rec=[915] Log Message=Delivering 22 to 23 rec=[916] Log Message=Delivering MCAST message with seq 23 to pending delivery queue rec=[917] Log Message=got joinlist message from node 3 rec=[918] Log Message=Received ringid(192.168.0.59:48964) seq 24 rec=[919] Log Message=Delivering 23 to 24 rec=[920] Log Message=Delivering MCAST message with seq 24 to pending delivery queue rec=[921] Log Message=Received ringid(192.168.0.59:48964) seq 25 rec=[922] Log Message=Delivering 24 to 25 rec=[923] Log Message=Delivering MCAST message with seq 25 to pending delivery queue rec=[924] Log Message=got joinlist message from node 4 rec=[925] Log Message=Received ringid(192.168.0.59:48964) seq 26 rec=[926] Log Message=Delivering 25 to 26 rec=[927] Log Message=Delivering MCAST message with seq 26 to pending delivery queue rec=[928] Log Message=Received ringid(192.168.0.59:48964) seq 27 rec=[929] Log Message=Delivering 26 to 27 rec=[930] Log Message=Delivering MCAST message with seq 27 to pending delivery queue rec=[931] Log Message=got joinlist message from node 2 rec=[932] Log Message=mcasted message added to pending queue rec=[933] Log Message=releasing messages up to and including 21 rec=[934] Log Message=Delivering 27 to 29 rec=[935] Log Message=Delivering MCAST message with seq 28 to pending delivery queue rec=[936] Log Message=Delivering MCAST message with seq 29 to pending delivery queue rec=[937] Log Message=comparing: sender r(0) ip(192.168.0.63) ; members(old:1 left:0) rec=[938] Log Message=comparing: sender r(0) ip(192.168.0.59) ; members(old:4 left:1) rec=[939] Log Message=comparing: sender r(0) ip(192.168.0.70) ; members(old:4 left:1) rec=[940] Log Message=comparing: sender r(0) ip(192.168.0.68) ; members(old:4 left:1) rec=[941] Log Message=chosen downlist: sender r(0) ip(192.168.0.59) ; members(old:4 left:1) rec=[942] Log Message=got joinlist message from node 1 rec=[943] Log Message=Received ringid(192.168.0.59:48964) seq 28 rec=[944] Log Message=Received ringid(192.168.0.59:48964) seq 29 rec=[945] Log Message=Received ringid(192.168.0.59:48964) seq 2a rec=[946] Log Message=Delivering 29 to 2a rec=[947] Log Message=Delivering MCAST message with seq 2a to pending delivery queue rec=[948] Log Message=confchg entries 4 rec=[949] Log Message=Barrier Start Received From 3 rec=[950] Log Message=Barrier completion status for nodeid 2 = 0. rec=[951] Log Message=Barrier completion status for nodeid 1 = 0. rec=[952] Log Message=Barrier completion status for nodeid 3 = 1. rec=[953] Log Message=Barrier completion status for nodeid 4 = 0. rec=[954] Log Message=Received ringid(192.168.0.59:48964) seq 2b rec=[955] Log Message=Delivering 2a to 2b rec=[956] Log Message=Delivering MCAST message with seq 2b to pending delivery queue rec=[957] Log Message=confchg entries 4 rec=[958] Log Message=Barrier Start Received From 4 rec=[959] Log Message=Barrier completion status for nodeid 2 = 0. rec=[960] Log Message=Barrier completion status for nodeid 1 = 0. rec=[961] Log Message=Barrier completion status for nodeid 3 = 1. rec=[962] Log Message=Barrier completion status for nodeid 4 = 1. rec=[963] Log Message=Received ringid(192.168.0.59:48964) seq 2c rec=[964] Log Message=Delivering 2b to 2c rec=[965] Log Message=Delivering MCAST message with seq 2c to pending delivery queue rec=[966] Log Message=confchg entries 4 rec=[967] Log Message=Barrier Start Received From 2 rec=[968] Log Message=Barrier completion status for nodeid 2 = 1. rec=[969] Log Message=Barrier completion status for nodeid 1 = 0. rec=[970] Log Message=Barrier completion status for nodeid 3 = 1. rec=[971] Log Message=Barrier completion status for nodeid 4 = 1. rec=[972] Log Message=mcasted message added to pending queue rec=[973] Log Message=releasing messages up to and including 27 rec=[974] Log Message=Delivering 2c to 2d rec=[975] Log Message=Delivering MCAST message with seq 2d to pending delivery queue rec=[976] Log Message=confchg entries 4 rec=[977] Log Message=Barrier Start Received From 1 rec=[978] Log Message=Barrier completion status for nodeid 2 = 1. rec=[979] Log Message=Barrier completion status for nodeid 1 = 1. rec=[980] Log Message=Barrier completion status for nodeid 3 = 1. rec=[981] Log Message=Barrier completion status for nodeid 4 = 1. rec=[982] Log Message=Synchronization barrier completed rec=[983] Log Message=Committing synchronization for (corosync cluster closed process group service v1.01) rec=[984] Log Message=Received ringid(192.168.0.59:48964) seq 2d rec=[985] Log Message=Received ringid(192.168.0.59:48964) seq 2e rec=[986] Log Message=Delivering 2d to 2e rec=[987] Log Message=Delivering MCAST message with seq 2e to pending delivery queue rec=[988] Log Message=Received ringid(192.168.0.59:48964) seq 2f rec=[989] Log Message=Delivering 2e to 2f rec=[990] Log Message=Delivering MCAST message with seq 2f to pending delivery queue rec=[991] Log Message=exit_fn for conn=0x17aee00 rec=[992] Log Message=Received ringid(192.168.0.59:48964) seq 30 rec=[993] Log Message=Delivering 2f to 30 rec=[994] Log Message=Delivering MCAST message with seq 30 to pending delivery queue rec=[995] Log Message=mcasted message added to pending queue rec=[996] Log Message=releasing messages up to and including 2c rec=[997] Log Message=Delivering 30 to 31 rec=[998] Log Message=Delivering MCAST message with seq 31 to pending delivery queue rec=[999] Log Message=Completed service synchronization, ready to provide service. rec=[1000] Log Message=got procleave message from cluster node 1 rec=[1001] Log Message=Received ringid(192.168.0.59:48964) seq 31 rec=[1002] Log Message=releasing messages up to and including 30 rec=[1003] Log Message=releasing messages up to and including 31 rec=[1004] Log Message=lib_init_fn: conn=0x17aee00 rec=[1005] Log Message=exit_fn for conn=0x17aee00 rec=[1006] Log Message=lib_init_fn: conn=0x17aee00 Finishing replay: records found [1006] I used corosync-1.3.4 and didn't have this problem Please,guide me to solve this problem Thanks in advance _______________________________________________ discuss mailing list discuss@xxxxxxxxxxxx http://lists.corosync.org/mailman/listinfo/discuss