Hi,
in this case I would recommend to use the ceph-bluestore-tool instead
of ceph-volume because it cleans up (removes the block.db symlink and
labels). These are the steps to revert that change:
# before reverting two devices are in use:
soc9-ceph:~ # ceph osd metadata 0 | grep devices
"bluefs_db_devices": "vdf",
"bluestore_bdev_devices": "vdb",
"devices": "vdb,vdf",
"objectstore_numa_unknown_devices": "vdb,vdf",
1. soc9-ceph:~ # ceph orch daemon stop osd.0
2. soc9-ceph:~ # cephadm shell --name osd.0
3. [ceph: root@soc9-ceph /]# ceph-bluestore-tool --path
/var/lib/ceph/osd/ceph-0/ --command bluefs-bdev-migrate --devs-source
/var/lib/ceph/osd/ceph-0/block.db --dev-target
/var/lib/ceph/osd/ceph-0/block
inferring bluefs devices from bluestore path
device removed:1 /var/lib/ceph/osd/ceph-0/block.db
4. IMPORTANT: Remove the DB LV before you start the OSD, otherwise the
OSD will start to use it again because of the LV tags. Alternatively,
delete the LV tags of the DB LV before starting the OSD.
5. soc9-ceph:~ # ceph orch daemon start osd.0
# check result
soc9-ceph:~ # ceph osd metadata 0 | grep devices
"bluestore_bdev_devices": "vdb",
"devices": "vdb",
"objectstore_numa_unknown_devices": "vdb",
Regards,
Eugen
Zitat von Jan Kasprzak <kas@xxxxxxxxxx>:
Hi all,
while reading a sibling thread about moving DB/WAL to a separate device,
I wonder whether is it possible to go the other way round as well,
i.e. to remove a metadata device from an OSD and merge metadata back
to the main storage?
What I am trying to do:
My OSD nodes are 1U boxes with 4 drive bays, two of which support NVMe.
They have two small-ish NVMe drives and two large HDDs.
On NVMe, there is a partition for an OS (configured as RAID-1 across both
NVMe drives), and the rest of each NVMe is used as metadata for one of the
HDD-based OSDs. So I have two OSDs per node.
Now I am considering replacing those small NVMes with much larger ones,
and use part of them for an OS as before, part of them as metadata for the
HDD-based OSDs as before, and the rest as new NVMe-only OSDs.
Can this be done without tearing the original HDD+NVMe-based OSDs down
and recreating them again?
Being able to remove a metadata device from the OSD would help in this case.
Thanks!
-Yenya
--
| Jan "Yenya" Kasprzak <kas at {fi.muni.cz - work | yenya.net - private}> |
| https://www.fi.muni.cz/~kas/ GPG: 4096R/A45477D5 |
We all agree on the necessity of compromise. We just can't agree on
when it's necessary to compromise. --Larry Wall
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
