On 26/11/14 09:24, Tan Ban Wee wrote:
Hi, This is a 2 nodes cluster and they are randomly rebooting itself. I hope someone can help me to narrow down to the cause. Nov 25 23:40:17 qdiskd Node 1 missed an update (3/4) Nov 25 23:40:18 qdiskd Node 1 missed an update (4/4) Nov 25 23:40:19 qdiskd Node 1 missed an update (5/4) Nov 25 23:40:19 qdiskd Node 1 DOWN Nov 25 23:40:19 qdiskd Writing eviction notice for node 1 Nov 25 23:40:19 qdiskd Telling CMAN to kill the node Nov 25 23:40:20 qdiskd Node 1 evicted Nov 25 23:44:19 qdiskd Node 1 is UP Nov 25 23:44:20 qdiskd Node 1 shutdown Nov 25 23:44:26 qdiskd Node 1 is UP Nov 25 23:44:37 qdiskd Node 1 missed an update (2/4) Nov 25 23:44:38 qdiskd Node 1 missed an update (3/4) Nov 25 23:44:39 qdiskd Node 1 missed an update (4/4) Nov 25 23:44:40 qdiskd Node 1 missed an update (5/4) Nov 25 23:44:40 qdiskd Node 1 DOWN Nov 25 23:44:40 qdiskd Writing eviction notice for node 1 Nov 25 23:44:40 qdiskd Telling CMAN to kill the node Nov 25 23:44:41 qdiskd Node 1 evicted Nov 25 23:50:48 qdiskd Loading dynamic configuration Nov 25 23:50:49 qdiskd Setting autocalculated votes to 1 Nov 25 23:50:49 qdiskd Loading static configuration Nov 25 23:50:49 qdiskd Auto-configured TKO as 4 based on token=10000 interval=1 Nov 25 23:50:49 qdiskd Timings: 4 tko, 1 interval Nov 25 23:50:49 qdiskd Timings: 2 tko_up, 3 master_wait, 2 upgrade_wait Nov 25 23:50:49 qdiskd Heuristic: 'ping -c3 -w5 10.101.210.250' score=1 interval=3 tko=5 Nov 25 23:50:49 qdiskd 1 heuristics loaded Nov 25 23:50:49 qdiskd Quorum Daemon: 1 heuristics, 1 interval, 4 tko, 1 votes Nov 25 23:50:49 qdiskd Run Flags: 00000231 Nov 25 23:50:49 qdiskd Header CRC32 mismatch; Exp: 0x00000000 Got: 0x190a55ad Nov 25 23:50:49 qdiskd diskRawReadShadow: bad CRC32, offset = 0 len = 512 Nov 25 23:50:49 qdiskd Header CRC32 mismatch; Exp: 0x00000000 Got: 0x190a55ad Nov 25 23:50:49 qdiskd diskRawReadShadow: bad CRC32, offset = 0 len = 512 Nov 25 23:50:49 qdiskd Header CRC32 mismatch; Exp: 0x00000000 Got: 0x190a55ad Nov 25 23:50:49 qdiskd diskRawReadShadow: bad CRC32, offset = 0 len = 512 Nov 25 23:50:49 qdiskd Header CRC32 mismatch; Exp: 0x00000000 Got: 0x190a55ad Nov 25 23:50:49 qdiskd diskRawReadShadow: bad CRC32, offset = 0 len = 512 Nov 25 23:50:49 qdiskd Header CRC32 mismatch; Exp: 0x00000000 Got: 0x190a55ad Nov 25 23:50:49 qdiskd diskRawReadShadow: bad CRC32, offset = 0 len = 512 Nov 25 23:50:49 qdiskd Header CRC32 mismatch; Exp: 0x00000000 Got: 0x190a55ad
From that limited information I would guess that your quorum disk partition is either offline or corrupted. First check that the drive is online and if it seems OK physically then check that it's not been formatted as a filesystem or something else by mistake and rebuild the header using mkqdisk.
Chrissie _______________________________________________ discuss mailing list discuss@xxxxxxxxxxxx http://lists.corosync.org/mailman/listinfo/discuss
