On fiveoften, ccsd starts up, but doesn't report its initial status (or connect to the cluster infrastructure) until after the error messages - 3min and 18sec after it has started up... At this point, I'm not sure what the hold-up is.
Once ccsd has reported that it is connected to the cluster infrastructure, everything is fine right?
I'll try to find out what the hold up is.
brassow
On Dec 10, 2004, at 5:05 PM, Matthew B. Brookover wrote:
On Fri, 2004-12-10 at 15:36, Jonathan E Brassow wrote:starting ccsd with the -v flag should print to the log the plugin it is
using to connect to the cluster infrastructure. Something on the order
of:
"Connected to cluster infrastructure via: CMAN/SM Plugin v1.1"
This will at least tell you that ccsd is able to connect to the cluster
manager - and therefore, should know whether the cluster is quorate or
not.
brassow
On Dec 10, 2004, at 4:16 PM, Matthew B. Brookover wrote:
On Fri, 2004-12-10 at 00:17, David Teigland wrote:This sounds similar to a problem I have if I run fence_tool without ccsd running.
Check /proc/cluster/status while it's waiting to see if the cluster
actually has quorum or not. Also, I've added some extra checking and
debugging to fence_tool that should help narrow down where things are
stuck. Please update from cvs and rebuild at least the stuff in
cluster/fence; then use "fence_tool join -D".
Usually things get stuck talking to ccs when ccs/magma libraries are out of sync, but this case sounds different.
I tried -v, and the cluster is quorate. Only the fencing is not starting up correctly.
Here are the logs with ccsd -v:
Added -v flag to ccsd, producing this log entry on fouroften:
Dec 10 15:50:00 fouroften kernel: Lock_Harness <CVS> (built Dec 10 2004
09:14:45) installed
Dec 10 15:50:00 fouroften kernel: GFS <CVS> (built Dec 10 2004 09:14:04)
installed
Dec 10 15:50:00 fouroften kernel: CMAN <CVS> (built Dec 10 2004
09:51:59) installed
Dec 10 15:50:00 fouroften kernel: NET: Registered protocol family 30
Dec 10 15:50:00 fouroften kernel: DLM <CVS> (built Dec 10 2004 09:52:25)
installed
Dec 10 15:50:00 fouroften kernel: Lock_DLM (built Dec 10 2004 09:14:25)
installed
Dec 10 15:50:00 fouroften ccsd[3379]: Starting ccsd DEVEL.1102700899:
Dec 10 15:50:01 fouroften ccsd[3379]: Built: Dec 10 2004 10:50:57
Dec 10 15:50:01 fouroften ccsd[3379]: Copyright (C) Red Hat, Inc.
2004 All rights reserved.
Dec 10 15:50:01 fouroften ccsd[3379]: Verbose Flag:: SET
Dec 10 15:50:01 fouroften ccsd[3379]: cluster.conf (cluster name =
CSMTEST, version = 6) found.
Dec 10 15:50:01 fouroften kernel: CMAN: Waiting to join or form a
Linux-cluster
Dec 10 15:50:01 fouroften crond(pam_unix)[3388]: session opened for user
root by (uid=0)
Dec 10 15:50:01 fouroften crond(pam_unix)[3387]: session opened for user
root by (uid=0)
Dec 10 15:50:02 fouroften ccsd[3379]: Connected to cluster infrastruture
via: CMAN/SM Plugin v1.1
Dec 10 15:50:02 fouroften ccsd[3379]: Initial status:: Inquorate
Dec 10 15:50:02 fouroften crond(pam_unix)[3388]: session closed for user
root
Dec 10 15:50:03 fouroften crond(pam_unix)[3387]: session closed for user
root
Dec 10 15:50:33 fouroften kernel: CMAN: forming a new cluster
Dec 10 15:50:33 fouroften kernel: CMAN: quorum regained, resuming
activity
Dec 10 15:50:33 fouroften kernel: CMAN: got node fiveoften
Logs on fiveoften:
Dec 10 15:50:07 fiveoften kernel: Lock_Harness <CVS> (built Dec 10 2004
09:14:45) installed
Dec 10 15:50:07 fiveoften kernel: GFS <CVS> (built Dec 10 2004 09:14:04)
installed
Dec 10 15:50:07 fiveoften crond(pam_unix)[3354]: session closed for user
root
Dec 10 15:50:07 fiveoften kernel: CMAN <CVS> (built Dec 10 2004
09:51:59) installed
Dec 10 15:50:07 fiveoften kernel: NET: Registered protocol family 30
Dec 10 15:50:07 fiveoften kernel: DLM <CVS> (built Dec 10 2004 09:52:25)
installed
Dec 10 15:50:07 fiveoften kernel: Lock_DLM (built Dec 10 2004 09:14:25)
installed
Dec 10 15:50:08 fiveoften ccsd[3381]: Starting ccsd DEVEL.1102700899:
Dec 10 15:50:08 fiveoften ccsd[3381]: Built: Dec 10 2004 10:50:57
Dec 10 15:50:08 fiveoften ccsd[3381]: Copyright (C) Red Hat, Inc.
2004 All rights reserved.
Dec 10 15:50:08 fiveoften ccsd[3381]: Verbose Flag:: SET
Dec 10 15:50:08 fiveoften ccsd[3381]: cluster.conf (cluster name =
CSMTEST, version = 6) found.
Dec 10 15:50:08 fiveoften kernel: CMAN: Waiting to join or form a
Linux-cluster
Dec 10 15:50:35 fiveoften kernel: CMAN: sending membership request
Dec 10 15:50:35 fiveoften kernel: CMAN: got node fouroften
Dec 10 15:50:35 fiveoften kernel: CMAN: quorum regained, resuming
activity
Dec 10 15:50:35 fiveoften ccsd[3381]: Cluster is not quorate. Refusing
connection.
Dec 10 15:50:35 fiveoften ccsd[3381]: Error while processing connect:
Connection refused
Dec 10 15:50:36 fiveoften ccsd[3381]: Cluster is not quorate. Refusing
connection.
Dec 10 15:50:36 fiveoften ccsd[3381]: Error while processing connect:
Connection refused
Dec 10 15:50:37 fiveoften ccsd[3381]: Cluster is not quorate. Refusing
connection.
Dec 10 15:50:37 fiveoften ccsd[3381]: Error while processing connect:
Connection refused
Dec 10 15:50:38 fiveoften ccsd[3381]: Cluster is not quorate. Refusing
connection.
Dec 10 15:50:38 fiveoften ccsd[3381]: Error while processing connect:
Connection refused
Dec 10 15:50:39 fiveoften ccsd[3381]: Cluster is not quorate. Refusing
connection.
Dec 10 15:50:39 fiveoften ccsd[3381]: Error while processing connect:
Connection refused
Dec 10 15:50:40 fiveoften ccsd[3381]: Cluster is not quorate. Refusing
connection.
Dec 10 15:50:40 fiveoften ccsd[3381]: Error while processing connect:
Connection refused
Dec 10 15:50:41 fiveoften ccsd[3381]: Cluster is not quorate. Refusing
connection.
Dec 10 15:50:41 fiveoften ccsd[3381]: Error while processing connect:
Connection refused
Dec 10 15:50:42 fiveoften ccsd[3381]: Cluster is not quorate. Refusing
connection.
Dec 10 15:50:42 fiveoften ccsd[3381]: Error while processing connect:
Connection refused
Dec 10 15:50:43 fiveoften ccsd[3381]: Cluster is not quorate. Refusing
connection.
Dec 10 15:50:43 fiveoften ccsd[3381]: Error while processing connect:
Connection refused
Dec 10 15:50:44 fiveoften ccsd[3381]: Cluster is not quorate. Refusing
connection.
Dec 10 15:50:44 fiveoften ccsd[3381]: Error while processing connect:
Connection refused
Dec 10 15:53:26 fiveoften ccsd[3381]: Connected to cluster infrastruture
via: CMAN/SM Plugin v1.1
Dec 10 15:53:26 fiveoften ccsd[3381]: Initial status:: Quorate
Matt mbrookov@xxxxxxxxx
-- Linux-cluster@xxxxxxxxxx http://www.redhat.com/mailman/listinfo/linux-cluster
