I have been experimenting with Ceph, and have some OSDs with drives containing XFS filesystems which I want to change to BTRFS. (I started with BTRFS, then started again from scratch with XFS [currently recommended] in order to eleminate that as a potential cause of some issues, now with further experience, I want to go back to BTRFS, but have data in my cluster and I don't want to scrap it). This is exactly equivalent to the case in which I have an OSD with a drive that I see is starting to error. I would in that case need to replace the drive and recreate the Ceph structures on it. So, I mark the OSD out, and the cluster automatically eliminates its notion of data stored on the OSD and creates copies of the affected PGs elsewhere to make the cluster healthy again. All of the disk replacement instructions that I see then tell me to then follow an OSD removal process: "This procedure removes an OSD from a cluster map, removes its authentication key, removes the OSD from the OSD map, and removes the OSD from the ceph.conf file". This seems to me to be too heavy-handed. I'm worried about doing this and then effectively adding a new OSD where I have the same id number as the OSD that I apparently unnecessarily removed. I don't actually want to remove the OSD. The OSD is fine, I just want to replace the disk drive that it uses. This suggests that I really want to take the OSD out, allow the cluster to get healthy again, then (replace the disk if this is due to failure,) create a new BTRFS/XFS filesystem, remount the drive, then recreate the Ceph structures on the disk to be compatible with the old disk and the original OSD that it was attached to. The OSD then gets marked back in, the cluster says "hello again, we missed you, but its good to see you back, here are some PGs ...". What I'm saying is that I really don't want to destroy the OSD, I want to refresh it with a new disk/filesystem and put it back to work. Is there some fundamental reason why this can't be done? If not, how should I do it? Best regards, David _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |