Hi Honza,
corosync and libqb are both upgraded to git master, and now they work
fine! Here is the new corosync.log, the [TOTEM ] ... came out.
[8905] toshiba corosyncnotice [MAIN ] Corosync Cluster Engine
('UNKNOWN'): started and ready to provide service.
[8905] toshiba corosyncinfo [MAIN ] Corosync built-in features:
pie relro bindnow
[8905] toshiba corosyncdebug [TOTEM ] waiting_trans_ack changed to 1
[8905] toshiba corosyncdebug [TOTEM ] Token Timeout (1000 ms)
retransmit timeout (238 ms)
[8905] toshiba corosyncdebug [TOTEM ] token hold (180 ms)
retransmits before loss (4 retrans)
[8905] toshiba corosyncdebug [TOTEM ] join (50 ms) send_join (0 ms)
consensus (1200 ms) merge (200 ms)
[8905] toshiba corosyncdebug [TOTEM ] downcheck (1000 ms) fail to
recv const (2500 msgs)
[8905] toshiba corosyncdebug [TOTEM ] seqno unchanged const (30
rotations) Maximum network MTU 1401
[8905] toshiba corosyncdebug [TOTEM ] window size per rotation (50
messages) maximum messages per rotation (17 messages)
[8905] toshiba corosyncdebug [TOTEM ] missed count const (5 messages)
[8905] toshiba corosyncdebug [TOTEM ] send threads (0 threads)
[8905] toshiba corosyncdebug [TOTEM ] RRP token expired timeout (238 ms)
[8905] toshiba corosyncdebug [TOTEM ] RRP token problem counter (2000 ms)
[8905] toshiba corosyncdebug [TOTEM ] RRP threshold (10 problem count)
[8905] toshiba corosyncdebug [TOTEM ] RRP multicast threshold (100
problem count)
[8905] toshiba corosyncdebug [TOTEM ] RRP automatic recovery check
timeout (1000 ms)
[8905] toshiba corosyncdebug [TOTEM ] RRP mode set to none.
[8905] toshiba corosyncdebug [TOTEM ] heartbeat_failures_allowed (0)
[8905] toshiba corosyncdebug [TOTEM ] max_network_delay (50 ms)
[8905] toshiba corosyncdebug [TOTEM ] HeartBeat is Disabled. To
enable set heartbeat_failures_allowed > 0
[8905] toshiba corosyncnotice [TOTEM ] Initializing transport (UDP/IP Unicast).
[8905] toshiba corosyncnotice [TOTEM ] Initializing transmit/receive
security (NSS) crypto: none hash: none
[8905] toshiba corosyncnotice [TOTEM ] The network interface
[192.168.1.12] is now up.
[8905] toshiba corosyncdebug [TOTEM ] Created or loaded sequence id
1a8.192.168.1.12 for this ring.
[8905] toshiba corosyncnotice [SERV ] Service engine loaded:
corosync configuration map access [0]
[8905] toshiba corosyncdebug [MAIN ] Initializing IPC on cmap [0]
[8905] toshiba corosyncdebug [MAIN ] No configured qb.ipc_type.
Using native ipc
[8905] toshiba corosyncinfo [QB ] server name: cmap
[8905] toshiba corosyncnotice [SERV ] Service engine loaded:
corosync configuration service [1]
[8905] toshiba corosyncdebug [MAIN ] Initializing IPC on cfg [1]
[8905] toshiba corosyncdebug [MAIN ] No configured qb.ipc_type.
Using native ipc
[8905] toshiba corosyncinfo [QB ] server name: cfg
[8905] toshiba corosyncnotice [SERV ] Service engine loaded:
corosync cluster closed process group service v1.01 [2]
[8905] toshiba corosyncdebug [MAIN ] Initializing IPC on cpg [2]
[8905] toshiba corosyncdebug [MAIN ] No configured qb.ipc_type.
Using native ipc
[8905] toshiba corosyncinfo [QB ] server name: cpg
[8905] toshiba corosyncnotice [SERV ] Service engine loaded:
corosync profile loading service [4]
[8905] toshiba corosyncdebug [MAIN ] NOT Initializing IPC on pload [4]
[8905] toshiba corosyncnotice [QUORUM] Using quorum provider
corosync_votequorum
[8905] toshiba corosyncdebug [VOTEQ ] Reading configuration (runtime: 0)
[8905] toshiba corosyncdebug [VOTEQ ] ev_tracking=0,
ev_tracking_barrier = 0: expected_votes = 0
[8905] toshiba corosyncdebug [VOTEQ ] total_votes=1, expected_votes=2
[8905] toshiba corosyncdebug [VOTEQ ] node 1 state=1, votes=1, expected=2
[8905] toshiba corosyncdebug [VOTEQ ] flags: quorate: No Leaving: No
WFA Status: No First: Yes Qdevice: No QdeviceAlive: No
QdeviceCastVote: No QdeviceMasterWins: No
[8905] toshiba corosyncnotice [SERV ] Service engine loaded:
corosync vote quorum service v1.0 [5]
[8905] toshiba corosyncdebug [MAIN ] Initializing IPC on votequorum [5]
[8905] toshiba corosyncdebug [MAIN ] No configured qb.ipc_type.
Using native ipc
[8905] toshiba corosyncinfo [QB ] server name: votequorum
[8905] toshiba corosyncnotice [SERV ] Service engine loaded:
corosync cluster quorum service v0.1 [3]
[8905] toshiba corosyncdebug [MAIN ] Initializing IPC on quorum [3]
[8905] toshiba corosyncdebug [MAIN ] No configured qb.ipc_type.
Using native ipc
[8905] toshiba corosyncinfo [QB ] server name: quorum
[8905] toshiba corosyncnotice [TOTEM ] adding new UDPU member {192.168.1.12}
[8905] toshiba corosyncnotice [TOTEM ] adding new UDPU member {192.168.1.13}
[8905] toshiba corosyncdebug [TOTEM ] entering GATHER state from
15(interface change).
[8905] toshiba corosyncdebug [TOTEM ] Creating commit token because
I am the rep.
[8905] toshiba corosyncdebug [TOTEM ] Saving state aru 0 high seq received 0
[8905] toshiba corosyncdebug [TOTEM ] Storing new sequence id for ring 1ac
[8905] toshiba corosyncdebug [TOTEM ] entering COMMIT state.
[8905] toshiba corosyncdebug [TOTEM ] got commit token
[8905] toshiba corosyncdebug [TOTEM ] entering RECOVERY state.
[8905] toshiba corosyncdebug [TOTEM ] position [0] member 192.168.1.12:
[8905] toshiba corosyncdebug [TOTEM ] previous ring seq 1a8 rep 192.168.1.12
[8905] toshiba corosyncdebug [TOTEM ] aru 0 high delivered 0 received flag 1
[8905] toshiba corosyncdebug [TOTEM ] Did not need to originate any
messages in recovery.
[8905] toshiba corosyncdebug [TOTEM ] got commit token
[8905] toshiba corosyncdebug [TOTEM ] Sending initial ORF token
[8905] toshiba corosyncdebug [TOTEM ] token retrans flag is 0 my set
retrans flag0 retrans queue empty 1 count 0, aru 0
[8905] toshiba corosyncdebug [TOTEM ] install seq 0 aru 0 high seq received 0
[8905] toshiba corosyncdebug [TOTEM ] token retrans flag is 0 my set
retrans flag0 retrans queue empty 1 count 1, aru 0
[8905] toshiba corosyncdebug [TOTEM ] install seq 0 aru 0 high seq received 0
[8905] toshiba corosyncdebug [TOTEM ] token retrans flag is 0 my set
retrans flag0 retrans queue empty 1 count 2, aru 0
[8905] toshiba corosyncdebug [TOTEM ] install seq 0 aru 0 high seq received 0
[8905] toshiba corosyncdebug [TOTEM ] token retrans flag is 0 my set
retrans flag0 retrans queue empty 1 count 3, aru 0
[8905] toshiba corosyncdebug [TOTEM ] install seq 0 aru 0 high seq received 0
[8905] toshiba corosyncdebug [TOTEM ] retrans flag count 4 token aru
0 install seq 0 aru 0 0
[8905] toshiba corosyncdebug [TOTEM ] Resetting old ring state
[8905] toshiba corosyncdebug [TOTEM ] recovery to regular 1-0
[8905] toshiba corosyncdebug [TOTEM ] waiting_trans_ack changed to 1
[8905] toshiba corosyncdebug [MAIN ] Member joined: r(0) ip(192.168.1.12)
[8905] toshiba corosyncdebug [TOTEM ] entering OPERATIONAL state.
[8905] toshiba corosyncnotice [TOTEM ] A new membership
(192.168.1.12:428) was formed. Members joined: 1
[8905] toshiba corosyncdebug [VOTEQ ] got nodeinfo message from cluster node 1
[8905] toshiba corosyncdebug [VOTEQ ] nodeinfo message[1]: votes: 1,
expected: 2 flags: 8
[8905] toshiba corosyncdebug [VOTEQ ] flags: quorate: No Leaving: No
WFA Status: No First: Yes Qdevice: No QdeviceAlive: No
QdeviceCastVote: No QdeviceMasterWins: No
[8905] toshiba corosyncdebug [VOTEQ ] total_votes=1, expected_votes=2
[8905] toshiba corosyncdebug [VOTEQ ] node 1 state=1, votes=1, expected=2
[8905] toshiba corosyncdebug [SYNC ] Committing synchronization for
corosync configuration map access
[8905] toshiba corosyncdebug [CMAP ] Single node sync -> no action
[8905] toshiba corosyncdebug [CPG ] comparing: sender r(0)
ip(192.168.1.12) ; members(old:0 left:0)
[8905] toshiba corosyncdebug [CPG ] chosen downlist: sender r(0)
ip(192.168.1.12) ; members(old:0 left:0)
[8905] toshiba corosyncdebug [SYNC ] Committing synchronization for
corosync cluster closed process group service v1.01
[8905] toshiba corosyncdebug [VOTEQ ] flags: quorate: No Leaving: No
WFA Status: No First: Yes Qdevice: No QdeviceAlive: No
QdeviceCastVote: No QdeviceMasterWins: No
[8905] toshiba corosyncdebug [VOTEQ ] got nodeinfo message from cluster node 1
[8905] toshiba corosyncdebug [VOTEQ ] nodeinfo message[1]: votes: 1,
expected: 2 flags: 8
[8905] toshiba corosyncdebug [VOTEQ ] flags: quorate: No Leaving: No
WFA Status: No First: Yes Qdevice: No QdeviceAlive: No
QdeviceCastVote: No QdeviceMasterWins: No
[8905] toshiba corosyncdebug [VOTEQ ] total_votes=1, expected_votes=2
[8905] toshiba corosyncdebug [VOTEQ ] node 1 state=1, votes=1, expected=2
[8905] toshiba corosyncdebug [VOTEQ ] got nodeinfo message from cluster node 1
[8905] toshiba corosyncdebug [VOTEQ ] nodeinfo message[0]: votes: 0,
expected: 0 flags: 0
[8905] toshiba corosyncdebug [SYNC ] Committing synchronization for
corosync vote quorum service v1.0
[8905] toshiba corosyncdebug [VOTEQ ] total_votes=1, expected_votes=2
[8905] toshiba corosyncdebug [VOTEQ ] node 1 state=1, votes=1, expected=2
[8905] toshiba corosyncnotice [QUORUM] Members[1]: 1
[8905] toshiba corosyncdebug [QUORUM] sending quorum notification to
(nil), length = 52
[8905] toshiba corosyncnotice [MAIN ] Completed service
synchronization, ready to provide service.
[8905] toshiba corosyncdebug [TOTEM ] waiting_trans_ack changed to 0
[8905] toshiba corosyncnotice [MAIN ] Node was shut down by a signal
[8905] toshiba corosyncnotice [SERV ] Unloading all Corosync service engines.
[8905] toshiba corosyncinfo [QB ] withdrawing server sockets
[8905] toshiba corosyncdebug [QB ] qb_ipcs_unref() - destroying
[8905] toshiba corosyncnotice [SERV ] Service engine unloaded:
corosync vote quorum service v1.0
[8905] toshiba corosyncinfo [QB ] withdrawing server sockets
[8905] toshiba corosyncdebug [QB ] qb_ipcs_unref() - destroying
[8905] toshiba corosyncnotice [SERV ] Service engine unloaded:
corosync configuration map access
[8905] toshiba corosyncinfo [QB ] withdrawing server sockets
[8905] toshiba corosyncdebug [QB ] qb_ipcs_unref() - destroying
[8905] toshiba corosyncnotice [SERV ] Service engine unloaded:
corosync configuration service
[8905] toshiba corosyncinfo [QB ] withdrawing server sockets
[8905] toshiba corosyncdebug [QB ] qb_ipcs_unref() - destroying
[8905] toshiba corosyncnotice [SERV ] Service engine unloaded:
corosync cluster closed process group service v1.01
[8905] toshiba corosyncinfo [QB ] withdrawing server sockets
[8905] toshiba corosyncdebug [QB ] qb_ipcs_unref() - destroying
[8905] toshiba corosyncnotice [SERV ] Service engine unloaded:
corosync cluster quorum service v0.1
[8905] toshiba corosyncnotice [SERV ] Service engine unloaded:
corosync profile loading service
[8905] toshiba corosyncdebug [TOTEM ] sending join/leave message
[8905] toshiba corosyncnotice [MAIN ] Corosync Cluster Engine exiting normally
Many thanks!
Just FYI, previous version I used is the RPM packages installed by rum
on Fedora 20:
[root@toshiba huzhj]# rpm -qi corosync
Name : corosync
Version : 2.3.3
Release : 1.fc20
Architecture: i686
Install Date: Wed 09 Apr 2014 10:24:33 PM CST
Group : System Environment/Base
Size : 423486
License : BSD
Signature : RSA/SHA256, Wed 15 Jan 2014 01:34:48 AM CST, Key ID
2eb161fa246110c1
Source RPM : corosync-2.3.3-1.fc20.src.rpm
Build Date : Tue 14 Jan 2014 11:54:10 PM CST
Build Host : buildvm-03.phx2.fedoraproject.org
Relocations : (not relocatable)
Packager : Fedora Project
Vendor : Fedora Project
URL : http://www.corosync.org/
Summary : The Corosync Cluster Engine and Application Programming Interfaces
Description :
This package contains the Corosync Cluster Engine Executive, several default
APIs and libraries, default configuration files, and an init script.
[root@toshiba huzhj]# rpm -qi libqb
Name : libqb
Version : 0.17.0
Release : 1.fc20
Architecture: i686
Install Date: Wed 09 Apr 2014 10:23:45 PM CST
Group : System Environment/Libraries
Size : 175458
License : LGPLv2+
Signature : RSA/SHA256, Fri 28 Feb 2014 02:57:16 AM CST, Key ID
2eb161fa246110c1
Source RPM : libqb-0.17.0-1.fc20.src.rpm
Build Date : Wed 26 Feb 2014 01:56:44 PM CST
Build Host : buildvm-15.phx2.fedoraproject.org
Relocations : (not relocatable)
Packager : Fedora Project
Vendor : Fedora Project
URL : http://www.libqb.org
Summary : An IPC library for high performance servers
Description :
libqb provides high performance client server reusable features.
Initially these are IPC and poll.
On Wed, May 28, 2014 at 11:42 PM, Jan Friesse <jfriesse@xxxxxxxxxx> wrote:
> Jason,
>
>> Hi Honza,
>>
>> I set to_logfile and logfile and this is my corosync.log:
>>
>> [2692] toshiba corosyncnotice [MAIN ] Corosync Cluster Engine
>> ('2.3.3'): started and ready to provide service.
>> [2692] toshiba corosyncinfo [MAIN ] Corosync built-in features:
>> dbus rdma systemd xmlconf snmp pie relro bindnow
>> [2692] toshiba corosyncnotice [SERV ] Service engine loaded:
>> corosync configuration map access [0]
>> [2692] toshiba corosyncdebug [MAIN ] Initializing IPC on cmap [0]
>> [2692] toshiba corosyncdebug [MAIN ] No configured qb.ipc_type.
>> Using native ipc
>> [2692] toshiba corosyncnotice [SERV ] Service engine loaded:
>> corosync configuration service [1]
>> [2692] toshiba corosyncdebug [MAIN ] Initializing IPC on cfg [1]
>> [2692] toshiba corosyncdebug [MAIN ] No configured qb.ipc_type.
>> Using native ipc
>> [2692] toshiba corosyncnotice [SERV ] Service engine loaded:
>> corosync cluster closed process group service v1.01 [2]
>> [2692] toshiba corosyncdebug [MAIN ] Initializing IPC on cpg [2]
>> [2692] toshiba corosyncdebug [MAIN ] No configured qb.ipc_type.
>> Using native ipc
>> [2692] toshiba corosyncnotice [SERV ] Service engine loaded:
>> corosync profile loading service [4]
>> [2692] toshiba corosyncdebug [MAIN ] NOT Initializing IPC on pload [4]
>> [2692] toshiba corosyncnotice [QUORUM] Using quorum provider
>> corosync_votequorum
>> [2692] toshiba corosyncdebug [QUORUM] Reading configuration (runtime: 0)
>> [2692] toshiba corosyncdebug [QUORUM] ev_tracking=0,
>> ev_tracking_barrier = 0: expected_votes = 0
>> [2692] toshiba corosyncdebug [QUORUM] total_votes=1, expected_votes=2
>> [2692] toshiba corosyncdebug [QUORUM] node 1 state=1, votes=1, expected=2
>> [2692] toshiba corosyncdebug [QUORUM] flags: quorate: No Leaving: No
>> WFA Status: No First: Yes Qdevice: No QdeviceAlive: No
>> QdeviceCastVote: No QdeviceMasterWins: No
>> [2692] toshiba corosyncnotice [SERV ] Service engine loaded:
>> corosync vote quorum service v1.0 [5]
>> [2692] toshiba corosyncdebug [MAIN ] Initializing IPC on votequorum [5]
>> [2692] toshiba corosyncdebug [MAIN ] No configured qb.ipc_type.
>> Using native ipc
>> [2692] toshiba corosyncnotice [SERV ] Service engine loaded:
>> corosync cluster quorum service v0.1 [3]
>> [2692] toshiba corosyncdebug [MAIN ] Initializing IPC on quorum [3]
>> [2692] toshiba corosyncdebug [MAIN ] No configured qb.ipc_type.
>> Using native ipc
>> [2692] toshiba corosyncdebug [MAIN ] Member joined: r(0) ip(192.168.1.12)
>> [2692] toshiba corosyncdebug [QUORUM] got nodeinfo message from cluster node 1
>> [2692] toshiba corosyncdebug [QUORUM] nodeinfo message[1]: votes: 1,
>> expected: 2 flags: 8
>> [2692] toshiba corosyncdebug [QUORUM] flags: quorate: No Leaving: No
>> WFA Status: No First: Yes Qdevice: No QdeviceAlive: No
>> QdeviceCastVote: No QdeviceMasterWins: No
>> [2692] toshiba corosyncdebug [QUORUM] total_votes=1, expected_votes=2
>> [2692] toshiba corosyncdebug [QUORUM] node 1 state=1, votes=1, expected=2
>> [2692] toshiba corosyncdebug [SYNC ] Committing synchronization for
>> corosync configuration map access
>> [2692] toshiba corosyncdebug [CMAP ] Single node sync -> no action
>> [2692] toshiba corosyncdebug [CPG ] comparing: sender r(0)
>> ip(192.168.1.12) ; members(old:0 left:0)
>> [2692] toshiba corosyncdebug [CPG ] chosen downlist: sender r(0)
>> ip(192.168.1.12) ; members(old:0 left:0)
>> [2692] toshiba corosyncdebug [SYNC ] Committing synchronization for
>> corosync cluster closed process group service v1.01
>> [2692] toshiba corosyncdebug [QUORUM] flags: quorate: No Leaving: No
>> WFA Status: No First: Yes Qdevice: No QdeviceAlive: No
>> QdeviceCastVote: No QdeviceMasterWins: No
>> [2692] toshiba corosyncdebug [QUORUM] got nodeinfo message from cluster node 1
>> [2692] toshiba corosyncdebug [QUORUM] nodeinfo message[1]: votes: 1,
>> expected: 2 flags: 8
>> [2692] toshiba corosyncdebug [QUORUM] flags: quorate: No Leaving: No
>> WFA Status: No First: Yes Qdevice: No QdeviceAlive: No
>> QdeviceCastVote: No QdeviceMasterWins: No
>> [2692] toshiba corosyncdebug [QUORUM] total_votes=1, expected_votes=2
>> [2692] toshiba corosyncdebug [QUORUM] node 1 state=1, votes=1, expected=2
>> [2692] toshiba corosyncdebug [QUORUM] got nodeinfo message from cluster node 1
>> [2692] toshiba corosyncdebug [QUORUM] nodeinfo message[0]: votes: 0,
>> expected: 0 flags: 0
>> [2692] toshiba corosyncdebug [SYNC ] Committing synchronization for
>> corosync vote quorum service v1.0
>> [2692] toshiba corosyncdebug [QUORUM] total_votes=1, expected_votes=2
>> [2692] toshiba corosyncdebug [QUORUM] node 1 state=1, votes=1, expected=2
>> [2692] toshiba corosyncnotice [QUORUM] Members[1]: 1
>> [2692] toshiba corosyncdebug [QUORUM] sending quorum notification to
>> (nil), length = 52
>> [2692] toshiba corosyncnotice [MAIN ] Completed service
>> synchronization, ready to provide service.
>> [2692] toshiba corosyncdebug [MAIN ] connection created
>> [2692] toshiba corosyncdebug [MAIN ] cs_ipcs_connection_closed()
>> [2692] toshiba corosyncdebug [MAIN ] cs_ipcs_connection_destroyed()
>>
>>
>>
>>
>> Still no [TOTEM] blah bla...... ,which printed by:
>>
>> log_printf (instance->totemsrp_log_level_debug, "..."); in totemsrp.c
>>
>>
>
> can you please try corosync from git master and libqb from git master?
> I'm remembering some strange issues with logging.
>
> Regards,
> Honza
>
>>
>> On Wed, May 28, 2014 at 10:43 PM, Jan Friesse <jfriesse@xxxxxxxxxx> wrote:
>>> Jason,
>>>
>>>> Hi All,
>>>> After switched from corosync-1.4.x to corosync-2.3.x, I can not get debug
>>>> log in exec/totem*.c any more. Set debug:on in logging section in
>>>> corosync.conf seems does not work as before(corosync-1.4.*). Only those sub
>>>> modules who use LOGSYS_DECLARE_SUBSYS macro still works. How can I get
>>>> totem debug log now?
>>>>
>>>
>>> I hope debug (global) still works as before. Only change which maybe you
>>> didn't hit before is move some of messages to trace level. This can be
>>> activated by setting debug: trace. Also keep in mind that default syslog
>>> is usually configured to not log debug messages, so it's better to set:
>>>
>>> to_logfile: yes
>>> logfile: /var/log/cluster/corosync.log
>>>
>>> and read messages from corosync.log.
>>>
>>> Regards,
>>> Honza
>>>
>>>> Below is my corosync.conf
>>>>
>>>>
>>>> totem {
>>>> version: 2
>>>> secauth: off
>>>> cluster_name: MyCluster
>>>> transport: udpu
>>>> }
>>>>
>>>> nodelist {
>>>> node {
>>>> ring0_addr: toshiba
>>>> nodeid: 1
>>>> }
>>>> node {
>>>> ring0_addr: vm1
>>>> nodeid: 2
>>>> }
>>>> }
>>>>
>>>> quorum {
>>>> provider: corosync_votequorum
>>>> }
>>>>
>>>> logging {
>>>> to_syslog: yes
>>>> debug: on
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> discuss mailing list
>>>> discuss@xxxxxxxxxxxx
>>>> http://lists.corosync.org/mailman/listinfo/discuss
>>>>
>>>
>>
>>
>>
>
--
Yours,
Jason
_______________________________________________
discuss mailing list
discuss@xxxxxxxxxxxx
http://lists.corosync.org/mailman/listinfo/discuss
