On 18/06/2019 08.12, Eitan Mosenkis wrote:
Hi.
I'm running a small single-host Ceph cluster on Proxmox (as my home
NAS). I want to encrypt my OSDs but I don't want the host's SSD to be a
single point of failure. What Ceph config/keyring/secret keys do I need
to make safe [encrypted] copies of to ensure that even if the host
running the Ceph monitor and OSDs experiences catastrophic data loss, I
can still recover the data on the OSD disks? Are the various keyring
files (ceph.mon.keyring, ceph.client.admin.keyring) sufficient?
Can I safely store an encrypted copy of that info on the lockbox
partition of each OSD or do I need to be concerned that Ceph will delete
unrecognized files from the lockbox?
Thanks!
I would recommend making your host's SSD a RAID1 pair or deploying two
monitors on different SSDs; this is what I do at home for a similar use
case, and should make recovery easier.
The problem is that if you lose the mon then your cluster is toast, and
backups don't help that much there because using an out-of-date backed
up mon with up-to-date OSDs is a recipe for disaster. The mon stores
critical state that must be in sync with the OSDs. Warping the mon back
in time is a bad idea. I asked about this very issue on the list in the
past :-)
There is a process for recovering the mon state from OSDs themselves:
http://docs.ceph.com/docs/mimic/rados/troubleshooting/troubleshooting-mon/#recovery-using-osds
And for that you would need at least the lockbox keys to be able to
decrypt the OSD partitions. You can use 'ceph config-key ls' to list all
your config keys; in my ceph-volume-on-lvm setup there are a bunch of
'dm-crypt/osd/<id>/luks' keys that hold the encryption passphrases you
need. Make sure you test that they work (not sure if they need to be
base64 decoded or what have you) if you really want to go this route.
--
Hector Martin (hector@xxxxxxxxxxxxxx)
Public Key: https://mrcn.st/pub
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
