But I am not using “cephx” for authentication. I have already disabled that. —Jiten On Nov 14, 2014, at 4:44 PM, Gregory Farnum <greg@xxxxxxxxxxx> wrote: > You didn't remove them from the auth monitor's keyring. If you're > removing OSDs you need to follow the steps in the documentation. > -Greg > > On Fri, Nov 14, 2014 at 4:42 PM, JIten Shah <jshah2005@xxxxxx> wrote: >> Hi Guys, >> >> I had to rekick some of the hosts where OSD’s were running and after >> re-kick, when I try to run puppet and install OSD’s again, it gives me a key >> mismatch error (as below). After the hosts were shutdown for rekick, I >> removed the OSD’s from the osd tree and the crush map too. Why is it still >> tied to the old key? >> >> >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> + test -b /osd-data >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> + mkdir -p /osd-data >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> + ceph-disk prepare /osd-data >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> + test -b /osd-data >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> + ceph-disk activate /osd-data >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> got monmap epoch 2 >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> 2014-11-15 00:31:25.951783 7ffff7fe67a0 -1 journal FileJournal::_open: >> disabling aio for non-block journal. Use journal_force_aio to force use of >> aio anyway >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> 2014-11-15 00:31:26.023037 7ffff7fe67a0 -1 journal FileJournal::_open: >> disabling aio for non-block journal. Use journal_force_aio to force use of >> aio anyway >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> 2014-11-15 00:31:26.023809 7ffff7fe67a0 -1 filestore(/osd-data) could not >> find 23c2fcde/osd_superblock/0//-1 in index: (2) No such file or directory >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> 2014-11-15 00:31:26.032044 7ffff7fe67a0 -1 created object store /osd-data >> journal /osd-data/journal for osd.2 fsid >> 2e738cda-1930-48cd-a4b1-74bc737c5d56 >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> 2014-11-15 00:31:26.032097 7ffff7fe67a0 -1 auth: error reading file: >> /osd-data/keyring: can't open /osd-data/keyring: (2) No such file or >> directory >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> 2014-11-15 00:31:26.032189 7ffff7fe67a0 -1 created new key in keyring >> /osd-data/keyring >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> Error EINVAL: entity osd.2 exists but key does not match >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> Traceback (most recent call last): >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> File "/usr/sbin/ceph-disk", line 2591, in <module> >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> main() >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> File "/usr/sbin/ceph-disk", line 2569, in main >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> args.func(args) >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> File "/usr/sbin/ceph-disk", line 1929, in main_activate >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> init=args.mark_init, >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> File "/usr/sbin/ceph-disk", line 1761, in activate_dir >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> (osd_id, cluster) = activate(path, activate_key_template, init) >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> File "/usr/sbin/ceph-disk", line 1897, in activate >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> keyring=keyring, >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> File "/usr/sbin/ceph-disk", line 1520, in auth_key >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> 'mon', 'allow profile osd', >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> File "/usr/sbin/ceph-disk", line 304, in command_check_call >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> return subprocess.check_call(arguments) >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> File "/usr/lib64/python2.6/subprocess.py", line 505, in check_call >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> raise CalledProcessError(retcode, cmd) >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> subprocess.CalledProcessError: Command '['/usr/bin/ceph', '--cluster', >> 'ceph', '--name', 'client.bootstrap-osd', '--keyring', >> '/var/lib/ceph/bootstrap-osd/ceph.keyring', 'auth', 'add', 'osd.2', '-i', >> '/osd-data/keyring', 'osd', 'allow *', 'mon', 'allow profile osd']' returned >> non-zero exit status 22 >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> + true >> Notice: >> /Stage[main]/Main/Node[infrastructure_node]/Ceph::Osd[/osd-data]/Exec[ceph-osd-mkfs-/osd-data]/returns: >> executed successfully >> >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |