If you want to be directly setting up the OSDs using ceph-volume commands (I'll pretty much always recommend following https://docs.ceph.com/en/latest/cephadm/services/osd/#dedicated-wal-db over manual ceph-volume stuff in cephadm deployments unless what you're doing can't be done with the spec files), you probably actually want to use `cephadm ceph-volume -- ...` rather than `cephadm shell`. The `cephadm ceph-volume` command mounts the provided keyring (or whatever keyring it infers) at `/var/lib/ceph/bootstrap-osd/ceph.keyring` inside the container, whereas the shell will not. So in theory you could try `cephadm ceph-volume -- lvm prepare --bluestore --data ceph-block-0/block-0 --block.db ceph-db-0/db-0` and that might get you past the keyring issue it seems to be complaining about. On Wed, Mar 6, 2024 at 2:10 PM <service.plant@xxxxx> wrote: > Hi all! > I;ve faced an issue I couldnt even google. > Trying to create OSD with two separate LVM for data.db and data, gives me > intresting error > > ``` > root@ceph-uvm2:/# ceph-volume lvm prepare --bluestore --data > ceph-block-0/block-0 --block.db ceph-db-0/db-0 > --> Incompatible flags were found, some values may get ignored > --> Cannot use None (None) with --bluestore (bluestore) > --> Incompatible flags were found, some values may get ignored > --> Cannot use --bluestore (bluestore) with --block.db (bluestore) > Running command: /usr/bin/ceph-authtool --gen-print-key > Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd > --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new > e3b7b9e5-6399-4e61-8634-41a991bb1948 > stderr: 2024-03-04T11:45:52.263+0000 7f71c3a89700 -1 auth: unable to find > a keyring on /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or > directory > stderr: 2024-03-04T11:45:52.263+0000 7f71c3a89700 -1 > AuthRegistry(0x7f71bc064978) no keyring found at > /var/lib/ceph/bootstrap-osd/ceph.keyring, disabling cephx > stderr: 2024-03-04T11:45:52.263+0000 7f71c3a89700 -1 auth: unable to find > a keyring on /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or > directory > stderr: 2024-03-04T11:45:52.263+0000 7f71c3a89700 -1 > AuthRegistry(0x7f71bc067fd0) no keyring found at > /var/lib/ceph/bootstrap-osd/ceph.keyring, disabling cephx > stderr: 2024-03-04T11:45:52.267+0000 7f71c3a89700 -1 auth: unable to find > a keyring on /var/lib/ceph/bootstrap-osd/ceph.keyring: (2) No such file or > directory > stderr: 2024-03-04T11:45:52.267+0000 7f71c3a89700 -1 > AuthRegistry(0x7f71c3a87ea0) no keyring found at > /var/lib/ceph/bootstrap-osd/ceph.keyring, disabling cephx > stderr: 2024-03-04T11:45:52.267+0000 7f71c1024700 -1 monclient(hunting): > handle_auth_bad_method server allowed_methods [2] but i only support [1] > stderr: 2024-03-04T11:45:52.267+0000 7f71c2026700 -1 monclient(hunting): > handle_auth_bad_method server allowed_methods [2] but i only support [1] > stderr: 2024-03-04T11:45:52.267+0000 7f71c1825700 -1 monclient(hunting): > handle_auth_bad_method server allowed_methods [2] but i only support [1] > stderr: 2024-03-04T11:45:52.267+0000 7f71c3a89700 -1 monclient: > authenticate NOTE: no keyring found; disabled cephx authentication > stderr: [errno 13] RADOS permission denied (error connecting to the > cluster) > --> RuntimeError: Unable to create a new OSD id > ``` > > And here is output which shows my created volumes > > ``` > root@ceph-uvm2:/# lvs > LV VG Attr LSize Pool Origin Data% Meta% Move > Log Cpy%Sync Convert > block-0 ceph-block-0 -wi-a----- 16.37t > > db-0 ceph-db-0 -wi-a----- 326.00g > ``` > > CEPH was rolled out by using cephadm and all the commands I do under > cephadm shell > > I have no idea what is the reason of that error and that is why I am here. > > > Any help is appreciated. > Thanks in advance. > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |