|
Hi, I have a more precise sequence in the syslog on the node entering the infernal loop "Node is undead", I think it could give some more indication about the problem, if someone could help me ... Thanks a lot Regards Alain [root@node3 ~]# tail -f /var/log/syslog /var/log/daemons/*& [1] 12017 [root@node3 ~]# ==> /var/log/syslog <== Mar 2 14:07:29 s_sys@node3 qdiskd[11100]: <info> Initial score 1/1 Mar 2 14:07:29 s_sys@node3 qdiskd[11100]: <info> Initialization complete Mar 2 14:07:29 s_sys@node3 openais[11209]: [CMAN ] quorum device registered Mar 2 14:07:29 s_sys@node3 qdiskd[11100]: <notice> Score sufficient for master operation (1/1; required=1); upgrading Mar 2 14:07:38 s_sys@node3 qdiskd[11100]: <info> Node 1 is the master ===> Here is the poweroff -nf on node2 Mar 2 14:07:59 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (2/16) Mar 2 14:08:00 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (3/16) Mar 2 14:08:01 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (4/16) Mar 2 14:08:02 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (5/16) Mar 2 14:08:03 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (6/16) ==> /var/log/daemons/errors <== Mar 2 13:18:24 s_sys@node3 ccsd[15202]: Error while processing connect: Connection refused Mar 2 13:18:24 s_sys@node3 ccsd[15202]: Error while processing connect: Connection refused Mar 2 13:18:25 s_sys@node3 ccsd[15202]: Error while processing connect: Connection refused Mar 2 13:18:25 s_sys@node3 ccsd[15202]: Error while processing connect: Connection refused Mar 2 13:46:31 s_sys@node3 mountd[16139]: Caught signal 15, un-registering and exiting. Mar 2 13:46:44 s_sys@node3 dlm_controld[15234]: cluster is down, exiting Mar 2 13:46:44 s_sys@node3 gfs_controld[15240]: cluster is down, exiting Mar 2 13:46:44 s_sys@node3 fenced[15228]: cluster is down, exiting Mar 2 13:46:44 s_sys@node3 qdiskd[15156]: <err> cman_dispatch: Host is down Mar 2 13:46:44 s_sys@node3 qdiskd[15156]: <err> Halting qdisk operations ==> /var/log/daemons/info <== Mar 2 14:06:58 s_sys@node3 openais[11209]: [MAIN ] Using default multicast address of 239.192.0.0 Mar 2 14:06:58 s_sys@node3 openais[11209]: [CMAN ] CMAN 2.0.98 (built Jan 27 2009 10:53:06) started Mar 2 14:06:59 s_sys@node3 openais[11209]: [CMAN ] quorum regained, resuming activity Mar 2 14:07:12 s_sys@node3 qdiskd[11100]: <info> Quorum Partition: /dev/disk/by-id/scsi-3600601607b801c00343ab689eb03de11 Label: QDISK_0_0 Mar 2 14:07:12 s_sys@node3 qdiskd[11100]: <info> Quorum Daemon Initializing Mar 2 14:07:13 s_sys@node3 qdiskd[11100]: <info> Heuristic: 'ping -W1 -c1 -t3 12.11.2.1; RES=$?; if [ $RES -ne 0 ]; then ping -W1 -c1 -t3 12.11.2.1; RES=$?; if [ $RES -ne 0 ]; then ping -W3 -c1 -t3 12.11.2.1; RES=$?; fi; fi; echo $RES | grep -w 0 ' UP Mar 2 14:07:29 s_sys@node3 qdiskd[11100]: <info> Initial score 1/1 Mar 2 14:07:29 s_sys@node3 qdiskd[11100]: <info> Initialization complete Mar 2 14:07:29 s_sys@node3 openais[11209]: [CMAN ] quorum device registered Mar 2 14:07:38 s_sys@node3 qdiskd[11100]: <info> Node 1 is the master ==> /var/log/daemons/warnings <== Mar 2 10:25:54 s_sys@node3 clurgmgrd: [5635]: <warning> Link for eth0: Not detected Mar 2 10:25:54 s_sys@node3 clurgmgrd: [5635]: <warning> No link on eth0... Mar 2 10:26:14 s_sys@node3 clurgmgrd: [5635]: <warning> Link for eth0: Not detected Mar 2 10:26:14 s_sys@node3 clurgmgrd: [5635]: <warning> No link on eth0... Mar 2 10:26:24 s_sys@node3 clurgmgrd: [5635]: <warning> Link for eth0: Not detected Mar 2 10:26:24 s_sys@node3 clurgmgrd: [5635]: <warning> No link on eth0... Mar 2 10:30:31 s_sys@node3 avahi-daemon[25488]: WARNING: No NSS support for mDNS detected, consider installing nss-mdns! Mar 2 11:11:15 s_sys@node3 avahi-daemon[25474]: WARNING: No NSS support for mDNS detected, consider installing nss-mdns! Mar 2 11:28:24 s_sys@node3 avahi-daemon[25466]: WARNING: No NSS support for mDNS detected, consider installing nss-mdns! Mar 2 13:56:47 s_sys@node3 avahi-daemon[25408]: WARNING: No NSS support for mDNS detected, consider installing nss-mdns! [root@node3 ~]# ==> /var/log/syslog <== Mar 2 14:08:04 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (7/16) Mar 2 14:08:05 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (8/16) Mar 2 14:08:06 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (9/16) Mar 2 14:08:07 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (10/16) Mar 2 14:08:08 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (11/16) Mar 2 14:08:09 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (12/16) Mar 2 14:08:10 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (13/16) Mar 2 14:08:11 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (14/16) Mar 2 14:08:12 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (15/16) Mar 2 14:08:13 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (16/16) Mar 2 14:08:14 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (17/16) Mar 2 14:08:14 s_sys@node3 qdiskd[11100]: <debug> Node 1 DOWN Mar 2 14:08:14 s_sys@node3 qdiskd[11100]: <debug> Making bid for master Mar 2 14:08:15 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (18/16) Mar 2 14:08:16 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (19/16) Mar 2 14:08:17 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (20/16) Mar 2 14:08:18 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (21/16) Mar 2 14:08:19 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (22/16) Mar 2 14:08:20 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (23/16) Mar 2 14:08:21 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (24/16) Mar 2 14:08:21 s_sys@node3 qdiskd[11100]: <info> Assuming master role ==> /var/log/daemons/info <== Mar 2 14:08:21 s_sys@node3 qdiskd[11100]: <info> Assuming master role ==> /var/log/syslog <== Mar 2 14:08:22 s_sys@node3 qdiskd[11100]: <debug> Node 1 missed an update (25/16) Mar 2 14:08:22 s_sys@node3 qdiskd[11100]: <notice> Writing eviction notice for node 1 Mar 2 14:08:22 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:23 s_sys@node3 qdiskd[11100]: <notice> Node 1 evicted Mar 2 14:08:24 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:24 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:24 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:25 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:25 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:25 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:26 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:26 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:26 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:27 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:27 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:27 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:28 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:28 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:28 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:29 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:29 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:29 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:29 s_sys@node3 openais[11209]: [TOTEM] The token was lost in the OPERATIONAL state. Mar 2 14:08:29 s_sys@node3 openais[11209]: [TOTEM] Receive multicast socket recv buffer size (288000 bytes). Mar 2 14:08:29 s_sys@node3 openais[11209]: [TOTEM] Transmit multicast socket send buffer size (262142 bytes). Mar 2 14:08:29 s_sys@node3 openais[11209]: [TOTEM] entering GATHER state from 2. Mar 2 14:08:30 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:30 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:30 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:31 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:31 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:31 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:32 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:32 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:32 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:33 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:33 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:33 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:34 s_sys@node3 openais[11209]: [TOTEM] entering GATHER state from 0. Mar 2 14:08:34 s_sys@node3 openais[11209]: [TOTEM] Creating commit token because I am the rep. Mar 2 14:08:34 s_sys@node3 openais[11209]: [TOTEM] Saving state aru 49 high seq received 49 Mar 2 14:08:34 s_sys@node3 openais[11209]: [TOTEM] Storing new sequence id for ring 27c Mar 2 14:08:34 s_sys@node3 openais[11209]: [TOTEM] entering COMMIT state. Mar 2 14:08:34 s_sys@node3 openais[11209]: [TOTEM] entering RECOVERY state. Mar 2 14:08:34 s_sys@node3 openais[11209]: [TOTEM] position [0] member 12.11.2.4: Mar 2 14:08:34 s_sys@node3 openais[11209]: [TOTEM] previous ring seq 632 rep 12.11.2.3 Mar 2 14:08:34 s_sys@node3 openais[11209]: [TOTEM] aru 49 high delivered 49 received flag 1 Mar 2 14:08:34 s_sys@node3 openais[11209]: [TOTEM] Did not need to originate any messages in recovery. Mar 2 14:08:34 s_sys@node3 openais[11209]: [TOTEM] Sending initial ORF token Mar 2 14:08:34 s_sys@node3 openais[11209]: [CLM ] CLM CONFIGURATION CHANGE Mar 2 14:08:34 s_sys@node3 openais[11209]: [CLM ] New Configuration: Mar 2 14:08:34 s_sys@node3 openais[11209]: [CLM ] r(0) ip(12.11.2.4) Mar 2 14:08:34 s_sys@node3 openais[11209]: [CLM ] Members Left: Mar 2 14:08:34 s_sys@node3 openais[11209]: [CLM ] r(0) ip(12.11.2.3) Mar 2 14:08:34 s_sys@node3 openais[11209]: [CLM ] Members Joined: Mar 2 14:08:34 s_sys@node3 openais[11209]: [cpg.c:0641] confchg, low nodeid=2, us = 2 Mar 2 14:08:34 s_sys@node3 openais[11209]: [cpg.c:0651] confchg, build downlist: 1 nodes Mar 2 14:08:34 s_sys@node3 openais[11209]: [CLM ] CLM CONFIGURATION CHANGE Mar 2 14:08:34 s_sys@node3 openais[11209]: [CLM ] New Configuration: Mar 2 14:08:34 s_sys@node3 openais[11209]: [CLM ] r(0) ip(12.11.2.4) Mar 2 14:08:34 s_sys@node3 openais[11209]: [CLM ] Members Left: Mar 2 14:08:34 s_sys@node3 openais[11209]: [CLM ] Members Joined: Mar 2 14:08:34 s_sys@node3 openais[11209]: [cpg.c:0662] confchg, sent downlist Mar 2 14:08:34 s_sys@node3 openais[11209]: [SYNC ] This node is within the primary component and will provide service. Mar 2 14:08:34 s_sys@node3 openais[11209]: [TOTEM] entering OPERATIONAL state. Mar 2 14:08:34 s_sys@node3 openais[11209]: [cpg.c:0785] downlist left_list: 1 Mar 2 14:08:34 s_sys@node3 openais[11209]: [cpg.c:0393] Sending new joinlist (1 elements) to clients Mar 2 14:08:34 s_sys@node3 openais[11209]: [cpg.c:0393] Sending new joinlist (1 elements) to clients Mar 2 14:08:34 s_sys@node3 openais[11209]: [cpg.c:0393] Sending new joinlist (1 elements) to clients Mar 2 14:08:34 s_sys@node3 openais[11209]: [cpg.c:0393] Sending new joinlist (1 elements) to clients Mar 2 14:08:34 s_sys@node3 openais[11209]: [CLM ] got nodejoin message 12.11.2.4 Mar 2 14:08:34 s_kernel@node3 kernel: dlm: closing connection to node 1 Mar 2 14:08:34 s_sys@node3 openais[11209]: [cpg.c:0959] sending joinlist to cluster Mar 2 14:08:34 s_sys@node3 openais[11209]: [CPG ] got joinlist message from node 2 Mar 2 14:08:34 s_sys@node3 openais[11209]: [cpg.c:1114] got mcast request on 0x128a57e0 Mar 2 14:08:34 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:34 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:34 s_sys@node3 openais[11209]: [cpg.c:1114] got mcast request on 0x128a57e0 Mar 2 14:08:34 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:34 s_sys@node3 fenced[11229]: node2 not a cluster member after 0 sec post_fail_delay Mar 2 14:08:34 s_sys@node3 fenced[11229]: fencing node "node2" ==> /var/log/daemons/info <== Mar 2 14:08:34 s_sys@node3 fenced[11229]: node2 not a cluster member after 0 sec post_fail_delay Mar 2 14:08:34 s_sys@node3 fenced[11229]: fencing node "node2" ==> /var/log/syslog <== Mar 2 14:08:35 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:35 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:35 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:36 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:36 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:36 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:37 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:37 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:37 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:38 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:38 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:38 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node [root@node3 ~]# Mar 2 14:08:39 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:39 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:39 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:39 s_sys@node3 fenced[11229]: fence "node2" success Mar 2 14:08:39 s_sys@node3 openais[11209]: [cpg.c:1114] got mcast request on 0x128a57e0 Mar 2 14:08:39 s_sys@node3 openais[11209]: [cpg.c:1114] got mcast request on 0x128a57e0 ==> /var/log/daemons/info <== Mar 2 14:08:39 s_sys@node3 fenced[11229]: fence "node2" success fg ==> /var/log/syslog <== Mar 2 14:08:40 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:40 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:40 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node tail -f /var/log/syslog /var/log/daemons/* [root@node3 ~]# tail -f /var/log/syslog /var/log/daemons/*& [1] 12044 [root@node3 ~]# ==> /var/log/syslog <== Mar 2 14:08:41 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:42 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:42 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:42 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:43 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:43 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:43 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:44 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:44 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:44 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node ==> /var/log/daemons/errors <== Mar 2 13:18:24 s_sys@node3 ccsd[15202]: Error while processing connect: Connection refused Mar 2 13:18:24 s_sys@node3 ccsd[15202]: Error while processing connect: Connection refused Mar 2 13:18:25 s_sys@node3 ccsd[15202]: Error while processing connect: Connection refused Mar 2 13:18:25 s_sys@node3 ccsd[15202]: Error while processing connect: Connection refused Mar 2 13:46:31 s_sys@node3 mountd[16139]: Caught signal 15, un-registering and exiting. Mar 2 13:46:44 s_sys@node3 dlm_controld[15234]: cluster is down, exiting Mar 2 13:46:44 s_sys@node3 gfs_controld[15240]: cluster is down, exiting Mar 2 13:46:44 s_sys@node3 fenced[15228]: cluster is down, exiting Mar 2 13:46:44 s_sys@node3 qdiskd[15156]: <err> cman_dispatch: Host is down Mar 2 13:46:44 s_sys@node3 qdiskd[15156]: <err> Halting qdisk operations ==> /var/log/daemons/info <== Mar 2 14:07:12 s_sys@node3 qdiskd[11100]: <info> Quorum Daemon Initializing Mar 2 14:07:13 s_sys@node3 qdiskd[11100]: <info> Heuristic: 'ping -W1 -c1 -t3 12.11.2.1; RES=$?; if [ $RES -ne 0 ]; then ping -W1 -c1 -t3 12.11.2.1; RES=$?; if [ $RES -ne 0 ]; then ping -W3 -c1 -t3 12.11.2.1; RES=$?; fi; fi; echo $RES | grep -w 0 ' UP Mar 2 14:07:29 s_sys@node3 qdiskd[11100]: <info> Initial score 1/1 Mar 2 14:07:29 s_sys@node3 qdiskd[11100]: <info> Initialization complete Mar 2 14:07:29 s_sys@node3 openais[11209]: [CMAN ] quorum device registered Mar 2 14:07:38 s_sys@node3 qdiskd[11100]: <info> Node 1 is the master Mar 2 14:08:21 s_sys@node3 qdiskd[11100]: <info> Assuming master role Mar 2 14:08:34 s_sys@node3 fenced[11229]: node2 not a cluster member after 0 sec post_fail_delay Mar 2 14:08:34 s_sys@node3 fenced[11229]: fencing node "node2" Mar 2 14:08:39 s_sys@node3 fenced[11229]: fence "node2" success ==> /var/log/daemons/warnings <== Mar 2 10:25:54 s_sys@node3 clurgmgrd: [5635]: <warning> Link for eth0: Not detected Mar 2 10:25:54 s_sys@node3 clurgmgrd: [5635]: <warning> No link on eth0... Mar 2 10:26:14 s_sys@node3 clurgmgrd: [5635]: <warning> Link for eth0: Not detected Mar 2 10:26:14 s_sys@node3 clurgmgrd: [5635]: <warning> No link on eth0... Mar 2 10:26:24 s_sys@node3 clurgmgrd: [5635]: <warning> Link for eth0: Not detected Mar 2 10:26:24 s_sys@node3 clurgmgrd: [5635]: <warning> No link on eth0... Mar 2 10:30:31 s_sys@node3 avahi-daemon[25488]: WARNING: No NSS support for mDNS detected, consider installing nss-mdns! Mar 2 11:11:15 s_sys@node3 avahi-daemon[25474]: WARNING: No NSS support for mDNS detected, consider installing nss-mdns! Mar 2 11:28:24 s_sys@node3 avahi-daemon[25466]: WARNING: No NSS support for mDNS detected, consider installing nss-mdns! Mar 2 13:56:47 s_sys@node3 avahi-daemon[25408]: WARNING: No NSS support for mDNS detected, consider installing nss-mdns! [root@node3 ~]# [root@node3 ~]# [root@node3 ~]# [root@node3 ~]# ==> /var/log/syslog <== Mar 2 14:08:45 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:45 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:45 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node tail -f /var/log/syslog /var/log/daemons/Mar 2 14:08:46 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:46 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:46 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN Mar 2 14:08:47 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:47 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:47 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:48 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:48 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:48 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:49 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:49 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:49 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:50 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:50 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:50 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:51 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:51 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:51 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:52 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:52 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:52 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:53 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:53 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:53 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:54 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:54 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:54 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:55 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:55 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:55 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:56 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:56 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:56 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:57 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:57 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:57 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:58 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:58 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:58 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:08:59 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:08:59 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:08:59 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:09:00 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:09:00 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:09:00 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:09:01 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:09:01 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:09:01 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:09:02 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:09:02 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 Mar 2 14:09:02 s_sys@node3 qdiskd[11100]: <debug> Telling CMAN to kill the node Mar 2 14:09:03 s_sys@node3 qdiskd[11100]: <crit> Node 1 is undead. Mar 2 14:09:03 s_sys@node3 qdiskd[11100]: <alert> Writing eviction notice for node 1 M |
-- Linux-cluster mailing list Linux-cluster@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/linux-cluster
