Hi,
we had a failing hard disk, and I replace it and want to create a new
OSD on it now.
But ceph-volume fails under these circumstances. In the original setup,
the OSDs were created with ceph-volume lvm batch using a bunch of drives
and a NVMe device for bluestore db. The batch mode uses a volume group
on the NVMe device instead of partitions.I have removed the former db
logical volume, the lvm setup for the former hard disk and all other
remainders. Creating a new OSD with any combination of devices now fails:
--data /dev/sda --block.db <nvme device>
--data /dev/sda --block.db <nvme volume group>
--data /dev/sda --block.db <lv created manually in nvme volume group>
# ceph-volume lvm create --bluestore --data /dev/sda --block.db
/dev/ceph-block-dbs-ea684aa8-544e-4c4a-8664-6cb50b3116b8/osd-block-db-a8f1489a-d97b-479e-b9a7-30fc9fa99cb5
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph --cluster ceph --name
client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring
-i - osd new 55cfb9f8-aa30-4f8b-8b95-a43d3f97fe5b
Running command: /sbin/vgcreate -s 1G --force --yes
ceph-1a2ddc14-780b-45e4-a036-e928862e6ccb /dev/sda
stdout: Physical volume "/dev/sda" successfully created.
stdout: Volume group "ceph-1a2ddc14-780b-45e4-a036-e928862e6ccb"
successfully created
Running command: /sbin/lvcreate --yes -l 100%FREE -n
osd-block-55cfb9f8-aa30-4f8b-8b95-a43d3f97fe5b
ceph-1a2ddc14-780b-45e4-a036-e928862e6ccb
stdout: Logical volume
"osd-block-55cfb9f8-aa30-4f8b-8b95-a43d3f97fe5b" created.
--> blkid could not detect a PARTUUID for device:
/dev/ceph-block-dbs-ea684aa8-544e-4c4a-8664-6cb50b3116b8/osd-block-db-a8f1489a-d97b-479e-b9a7-30fc9fa99cb5
--> Was unable to complete a new OSD, will rollback changes
Running command: /usr/bin/ceph --cluster ceph --name
client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring
osd purge-new osd.136 --yes-i-really-mean-it
stderr: purged osd.136
--> RuntimeError: unable to use device
In all cases ceph-volume is not able to detect a partition uuid for the
db device (which is correct, since the device is a logical volume....).
Running 'ceph-volume lvm batch' again results in a OSD without using the
NVMe device as db.
So what is the recommended way to manually create an OSD with a certain
hard disk and an existing logical volume as db partition? I would like
to avoid to zap all other OSDs using the NVMe device and recreate them
in a single run with 'ceph-volume lvm batch ...'.
Regards,
Burkhard
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
