[ Re-adding users list ] On Thu, Dec 12, 2019 at 10:12 AM Rodrigo Severo - Fábrica <rodrigo@xxxxxxxxxxxxxxxxxxx> wrote: > > Em qui., 12 de dez. de 2019 às 12:55, Gregory Farnum > <gfarnum@xxxxxxxxxx> escreveu: > > > > On Wed, Dec 11, 2019 at 12:04 PM Rodrigo Severo - Fábrica <rodrigo@xxxxxxxxxxxxxxxxxxx> wrote: > >> > >> Hi, > >> > >> > >> Trying to create a new OSD following the instructions available at > >> https://docs.ceph.com/docs/master/rados/operations/add-or-rm-osds/ > >> > >> On step 3 I'm instructed to run "ceph-osd -i {osd-num} --mkfs > >> --mkkey". Unfortunately it doesn't work: > >> > >> # ceph-osd -i 3 --mkfs --mkkey > >> 2019-12-11 16:59:58.257 7fac4597fc00 -1 auth: unable to find a keyring > >> on /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory > >> 2019-12-11 16:59:58.257 7fac4597fc00 -1 AuthRegistry(0x55ad976ea140) > >> no keyring found at /var/lib/ceph/osd/ceph-3/keyring, disabling cephx > >> 2019-12-11 16:59:58.261 7fac4597fc00 -1 auth: unable to find a keyring > >> on /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory > >> 2019-12-11 16:59:58.261 7fac4597fc00 -1 AuthRegistry(0x7fffac4075e8) > >> no keyring found at /var/lib/ceph/osd/ceph-3/keyring, disabling cephx > >> failed to fetch mon config (--no-mon-config to skip) > > I'm full of questions. > > > This is the important bit. Since you’re running the command without a way to access the Ceph monitors, > > Should I provide a way do access the Ceph monitors? I was just > following the docs and there is no mention to it in the above page. > > Anyway I tried to do it but the results were exactly the same: > > # ceph-osd -m 192.168.109.233:3300 -i 3 --mkfs --mkkey > 2019-12-12 14:56:18.408 7f392c0a1c00 -1 auth: unable to find a keyring > on /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory > 2019-12-12 14:56:18.408 7f392c0a1c00 -1 AuthRegistry(0x564545484140) > no keyring found at /var/lib/ceph/osd/ceph-3/keyring, disabling cephx > 2019-12-12 14:56:18.408 7f392c0a1c00 -1 auth: unable to find a keyring > on /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory > 2019-12-12 14:56:18.408 7f392c0a1c00 -1 AuthRegistry(0x7ffccdef0958) > no keyring found at /var/lib/ceph/osd/ceph-3/keyring, disabling cephx > failed to fetch mon config (--no-mon-config to skip) > > I also tried in the 6789 port and without providing port at all. > Always the same results. Tried also on the other monitors. Always the > same results. > > > it can’t use the cluster default configuration. You need to move the client.admin keyring onto the node (or another one, if you configure the ceph.conf or pass in the name on each command invocation). I haven't been through the docs in a while so maybe it's missing or lost but it should describe that somewhere. This is necessary whenever you are making changes to the cluster, such as adding an OSD — without a keyring, the cluster has no idea if the user making the change is authorized! > > Shouldn't it automatically read /etc/ceph/ceph.conf and figure everything out? > > I also tried explicitly setting the conf path but, again, same results: It's reading the ceph.conf but still needs cluster access, which you haven't given it. > > # ceph-osd -i 3 --mkfs --mkkey -c /etc/ceph/ceph.conf > 2019-12-12 14:57:35.740 7fee29b61c00 -1 auth: unable to find a keyring > on /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory > 2019-12-12 14:57:35.740 7fee29b61c00 -1 AuthRegistry(0x56457c784140) > no keyring found at /var/lib/ceph/osd/ceph-3/keyring, disabling cephx > 2019-12-12 14:57:35.740 7fee29b61c00 -1 auth: unable to find a keyring > on /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory > 2019-12-12 14:57:35.740 7fee29b61c00 -1 AuthRegistry(0x7ffc6eccb0b8) > no keyring found at /var/lib/ceph/osd/ceph-3/keyring, disabling cephx > failed to fetch mon config (--no-mon-config to skip) > > > You can pass the given option to not worry about that, > > Tried that. Apparently it worked, despite several disturbing messages: > > # ceph-osd -i 3 --mkfs --mkkey --no-mon-config > 2019-12-12 14:57:50.256 7f31d77b2c00 -1 auth: error reading file: > /var/lib/ceph/osd/ceph-3/keyring: can't open > /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory > 2019-12-12 14:57:50.260 7f31d77b2c00 -1 created new key in keyring > /var/lib/ceph/osd/ceph-3/keyring > 2019-12-12 14:57:50.260 7f31d77b2c00 -1 > bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to > open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory > 2019-12-12 14:57:50.260 7f31d77b2c00 -1 > bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to > open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory > 2019-12-12 14:57:50.260 7f31d77b2c00 -1 > bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to > open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory > 2019-12-12 14:57:50.629 7f31d77b2c00 -1 > bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid > # ll /var/lib/ceph/osd/ceph-3 > total 172 > drwxr-xr-x 2 root root 4096 Dec 12 14:57 ./ > drwxr-xr-x 4 ceph ceph 4096 Dec 11 16:19 ../ > -rw-r----- 1 root root 10737418240 Dec 12 14:57 block > -rw------- 1 root root 2 Dec 12 14:57 bluefs > -rw------- 1 root root 37 Dec 12 14:57 ceph_fsid > -rw-r----- 1 root root 37 Dec 12 14:57 fsid > -rw------- 1 root root 56 Dec 12 14:57 keyring > -rw------- 1 root root 8 Dec 12 14:57 kv_backend > -rw------- 1 root root 21 Dec 12 14:57 magic > -rw------- 1 root root 4 Dec 12 14:57 mkfs_done > -rw------- 1 root root 6 Dec 12 14:57 ready > -rw------- 1 root root 10 Dec 12 14:57 type > -rw------- 1 root root 2 Dec 12 14:57 whoami > > Is it how it should be? With all these "failed to open" messages? Yeah, those are BlueStore et al attempting various opens that are allowed to fail. I agree they're not great to have show up as standard. > > > or you can run the command with a user key ring that’s allowed to read the monitor configs. (You’ll need that key access next to add the OSD’s key to the cluster, anyway!) > > Haven't tried that. How is it done? I can't find any option to provide > a user key ring in ceph-osd's man page. It's not ceph-osd specific, just anywhere in ceph. -n client.[name], -k [keyring_file] etc > > > Many thanks for your help. > > Rodrigo > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |