On Mon, Jun 4, 2018 at 7:09 AM, Wido den Hollander <wido@xxxxxxxx> wrote: > Hi, > > After getting back from vacation the first thing to do is deploying a > Mimic cluster and I ran into some problems. > > For this very test setup I'm using VMs and ceph-deploy. > > This command however fails: > > $ ceph-deploy osd create bravo --data /dev/sdb > > [bravo][DEBUG ] Running command: /bin/ln -s > /dev/ceph-46188fa4-f30f-4e87-8dce-6027dc851108/osd-block-6e9ba998-e69a-42a9-911c-5ed8a31fba1f > /var/lib/ceph/osd/ceph-1/block > [bravo][DEBUG ] Running command: /usr/bin/ceph --cluster ceph --name > client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring > mon getmap -o /var/lib/ceph/osd/ceph-1/activate.monmap > [bravo][DEBUG ] stderr: got monmap epoch 1 > [bravo][DEBUG ] Running command: /usr/bin/ceph-authtool > /var/lib/ceph/osd/ceph-1/keyring --create-keyring --name osd.1 --add-key > AQCSEBVb8OAJNBAAK0DOoQpLdDz2WjW4YvG/CA== > [bravo][DEBUG ] stdout: creating /var/lib/ceph/osd/ceph-1/keyring > [bravo][DEBUG ] stdout: added entity osd.1 auth auth(auid = > 18446744073709551615 key=AQCSEBVb8OAJNBAAK0DOoQpLdDz2WjW4YvG/CA== with 0 > caps) > [bravo][DEBUG ] Running command: /bin/chown -R ceph:ceph > /var/lib/ceph/osd/ceph-1/keyring > [bravo][DEBUG ] Running command: /bin/chown -R ceph:ceph > /var/lib/ceph/osd/ceph-1/ > [bravo][DEBUG ] Running command: /usr/bin/ceph-osd --cluster ceph > --osd-objectstore bluestore --mkfs -i 1 --monmap > /var/lib/ceph/osd/ceph-1/activate.monmap --keyfile - --osd-data > /var/lib/ceph/osd/ceph-1/ --osd-uuid > 6e9ba998-e69a-42a9-911c-5ed8a31fba1f --setuser ceph --setgroup ceph > [bravo][DEBUG ] --> ceph-volume lvm prepare successful for: /dev/sdb > [bravo][DEBUG ] Running command: /usr/bin/ceph-bluestore-tool > --cluster=ceph prime-osd-dir --dev > /dev/ceph-46188fa4-f30f-4e87-8dce-6027dc851108/osd-block-6e9ba998-e69a-42a9-911c-5ed8a31fba1f > --path /var/lib/ceph/osd/ceph-1 > [bravo][DEBUG ] stderr: 2018-06-04 12:12:52.523 7f0a444e1a00 -1 auth: > unable to find a keyring on > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: > (2) No such file or directory > [bravo][DEBUG ] stderr: 2018-06-04 12:12:52.523 7f0a444e1a00 -1 > monclient: ERROR: missing keyring, cannot use cephx for authentication > [bravo][DEBUG ] stderr: failed to fetch mon config (--no-mon-config to > skip) > > The lines get wrapped by the e-mail, but this is the failing command: > > /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev > /dev/ceph-46188fa4-f30f-4e87-8dce-6027dc851108/osd-block-6e9ba998-e69a-42a9-911c-5ed8a31fba1f > --path /var/lib/ceph/osd/ceph-1 > > 2018-06-04 13:05:08.611 7fb66d34ba00 -1 auth: unable to find a keyring > on > /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: > (2) No such file or directory > 2018-06-04 13:05:08.619 7fb66d34ba00 -1 monclient: ERROR: missing > keyring, cannot use cephx for authentication > failed to fetch mon config (--no-mon-config to skip) > > The */var/lib/ceph/bootstrap-osd/ceph.keyring* file is present and has > the proper content. > > If I put */etc/ceph/ceph.client.admin.keyring* on that system the > deployment of the OSD works, but that shouldn't be right. > > I manually tried to supply some arguments to *ceph-bluestore-tool* for > the sake of testing, but that didn't work either. > > Now, if I add '--no-mon-config' to *ceph-bluestore-tool* it primes the > directory just fine. > > So is this a bug in ceph-volume? I assume so, but I wanted to > double-check before submitting a PR and issue. This is a problem in ceph-deploy, not ceph-volume ceph-volume is tested using ceph-ansible only, and there are no issues there. This is probably due to the recent changes in master/mimic regarding keys > > OS: Ubuntu 16.04.4 > Ceph: 13.2.0 (79a10589f1f80dfe21e8f9794365ed98143071c4) mimic (stable) > > Wido > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
