adding ceph mon with ceph-deply ends in ceph-create-keys:ceph-mon is not in quorum: u'probing' / monmap with 0.0.0.0:0 addresses

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

ceph-deploy mon create ceph5

builds the monitor ( 5th new monitor, 4 already existing )

ceph5# python /usr/sbin/ceph-create-keys --cluster ceph -i ceph5

hangs with:

INFO:ceph-create-keys:ceph-mon is not in quorum: u'probing'
INFO:ceph-create-keys:ceph-mon is not in quorum: u'probing'
^CTraceback (most recent call last):
  File "/usr/sbin/ceph-create-keys", line 231, in <module>
    main()
  File "/usr/sbin/ceph-create-keys", line 213, in main
    wait_for_quorum(cluster=args.cluster, mon_id=args.id)
  File "/usr/sbin/ceph-create-keys", line 51, in wait_for_quorum
    time.sleep(1)
KeyboardInterrupt

-------------

ceph5# python /bin/ceph --cluster=ceph --admin-daemon=/var/run/ceph/ceph-mon.ceph5.asok mon_status

Results in:

{
    "name": "ceph5",
    "rank": -1,
    "state": "probing",
    "election_epoch": 0,
    "quorum": [],
    "outside_quorum": [],
    "extra_probe_peers": [
        "10.0.0.1:6789\/0",
        "10.0.0.2:6789\/0",
        "10.0.0.3:6789\/0",
        "10.0.0.4:6789\/0"
    ],
    "sync_provider": [],
    "monmap": {
        "epoch": 0,
        "fsid": "97280f73-490b-4fb3-a9ea-f6a635140abc",
        "modified": "0.000000",
        "created": "0.000000",
        "mons": [
            {
                "rank": 0,
                "name": "ceph1",
                "addr": "0.0.0.0:0\/1"
            },
            {
                "rank": 1,
                "name": "ceph2",
                "addr": "0.0.0.0:0\/2"
            },
            {
                "rank": 2,
                "name": "ceph3",
                "addr": "0.0.0.0:0\/3"
            },
            {
                "rank": 3,
                "name": "ceph4",
                "addr": "0.0.0.0:0\/4"
            }
        ]
    }
}


While

"extra_probe_peers": [
"10.0.0.1:6789\/0",
"10.0.0.2:6789\/0",
"10.0.0.3:6789\/0",
"10.0.0.4:6789\/0"

is correct.


0.0.0.0:0\/1
0.0.0.0:0\/2
0.0.0.0:0\/3
0.0.0.0:0\/4

is obviously not.

The Logs says:

2015-11-05 23:05:53.417731 7f17278b7700 1 -- 10.0.0.55:6789/0 --> mon.0 0.0.0.0:0/1 -- mon_probe(probe 97280f73-490b-4fb3-a9ea-f6a635140abc name ceph5 new) v6 -- ?+0 0x530e400 2015-11-05 23:05:53.417751 7f17278b7700 1 -- 10.0.0.55:6789/0 --> mon.1 0.0.0.0:0/2 -- mon_probe(probe 97280f73-490b-4fb3-a9ea-f6a635140abc name ceph5 new) v6 -- ?+0 0x530e180 2015-11-05 23:05:53.417770 7f17278b7700 1 -- 10.0.0.55:6789/0 --> mon.2 0.0.0.0:0/3 -- mon_probe(probe 97280f73-490b-4fb3-a9ea-f6a635140abc name ceph5 new) v6 -- ?+0 0x530df00 2015-11-05 23:05:53.417800 7f17278b7700 1 -- 10.0.0.55:6789/0 --> mon.3 0.0.0.0:0/4 -- mon_probe(probe 97280f73-490b-4fb3-a9ea-f6a635140abc name ceph5 new) v6 -- ?+0 0x530dc80 2015-11-05 23:05:53.417820 7f17278b7700 1 -- 10.0.0.55:6789/0 --> mon.? 10.0.0.1:6789/0 -- mon_probe(probe 97280f73-490b-4fb3-a9ea-f6a635140abc name ceph5 new) v6 -- ?+0 0x530da00 2015-11-05 23:05:53.417839 7f17278b7700 1 -- 10.0.0.55:6789/0 --> mon.? 10.0.0.2:6789/0 -- mon_probe(probe 97280f73-490b-4fb3-a9ea-f6a635140abc name ceph5 new) v6 -- ?+0 0x530d780 2015-11-05 23:05:53.417863 7f17278b7700 1 -- 10.0.0.55:6789/0 --> mon.? 10.0.0.3:6789/0 -- mon_probe(probe 97280f73-490b-4fb3-a9ea-f6a635140abc name ceph5 new) v6 -- ?+0 0x530d500 2015-11-05 23:05:53.417883 7f17278b7700 1 -- 10.0.0.55:6789/0 --> mon.? 10.0.0.4:6789/0 -- mon_probe(probe 97280f73-490b-4fb3-a9ea-f6a635140abc name ceph5 new) v6 -- ?+0 0x530d280


10.0.0.55 is ceph5

Also cephx verify_reply problems:

2015-11-05 23:05:53.419746 7f17254af700 0 cephx: verify_reply couldn't decrypt with error: error decoding block for decryption 2015-11-05 23:05:53.419767 7f17254af700 0 -- 10.0.0.55:6789/0 >> 10.0.0.2:6789/0 pipe(0x49b1000 sd=19 :58188 s=1 pgs=0 cs=0 l=0 c=0x473aec0).failed verifying authorize reply


ceph-s is fine, no clock screw, no nothing, all fine.


Any suggestions ?

Thank you !


--
Mit freundlichen Gruessen / Best regards

Oliver Dzombic

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux