| Hi Guys, I had to rekick some of the hosts where OSD’s were running and after re-kick, when I try to run puppet and install OSD’s again, it gives me a key mismatch error (as below). After the hosts were shutdown for rekick, I removed the OSD’s from the osd tree and the crush map too. Why is it still tied to the old key? Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: + test -b /osd-data Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: + mkdir -p /osd-data Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: + ceph-disk prepare /osd-data Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: + test -b /osd-data Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: + ceph-disk activate /osd-data Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: got monmap epoch 2 Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: 2014-11-15 00:31:25.951783 7ffff7fe67a0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: 2014-11-15 00:31:26.023037 7ffff7fe67a0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: 2014-11-15 00:31:26.023809 7ffff7fe67a0 -1 filestore(/osd-data) could not find 23c2fcde/osd_superblock/0//-1 in index: (2) No such file or directory Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: 2014-11-15 00:31:26.032044 7ffff7fe67a0 -1 created object store /osd-data journal /osd-data/journal for osd.2 fsid 2e738cda-1930-48cd-a4b1-74bc737c5d56 Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: 2014-11-15 00:31:26.032097 7ffff7fe67a0 -1 auth: error reading file: /osd-data/keyring: can't open /osd-data/keyring: (2) No such file or directory Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: 2014-11-15 00:31:26.032189 7ffff7fe67a0 -1 created new key in keyring /osd-data/keyring Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: Error EINVAL: entity osd.2 exists but key does not match Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: Traceback (most recent call last): Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: File "/usr/sbin/ceph-disk", line 2591, in <module> Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: main() Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: File "/usr/sbin/ceph-disk", line 2569, in main Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: args.func(args) Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: File "/usr/sbin/ceph-disk", line 1929, in main_activate Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: init=args.mark_init, Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: File "/usr/sbin/ceph-disk", line 1761, in activate_dir Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: (osd_id, cluster) = activate(path, activate_key_template, init) Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: File "/usr/sbin/ceph-disk", line 1897, in activate Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: keyring=keyring, Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: File "/usr/sbin/ceph-disk", line 1520, in auth_key Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: 'mon', 'allow profile osd', Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: File "/usr/sbin/ceph-disk", line 304, in command_check_call Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: return subprocess.check_call(arguments) Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: File "/usr/lib64/python2.6/subprocess.py", line 505, in check_call Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: raise CalledProcessError(retcode, cmd) Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: subprocess.CalledProcessError: Command '['/usr/bin/ceph', '--cluster', 'ceph', '--name', 'client.bootstrap-osd', '--keyring', '/var/lib/ceph/bootstrap-osd/ceph.keyring', 'auth', 'add', 'osd.2', '-i', '/osd-data/keyring', 'osd', 'allow *', 'mon', 'allow profile osd']' returned non-zero exit status 22 Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: + true Notice: /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: executed successfully |
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
