On Sat, May 26, 2018 at 11:51 AM Bryan Henderson <bryanh@xxxxxxxxxxxxxxxx> wrote:
>> Suppose I lost all monitors in a ceph cluster in my laboratory. I have
>> all OSDs intact. Is it possible to recover something from Ceph?
>
>Yes, there is. Using ceph-objectstore-tool you are able to rebuild the
>MON database.
>
>BUT, this isn't something you would really want to do as you loose your
>cephx keys and such and getting them all back will be a total nightmare.
According to the section of the manual on this, TROUBLESHOOTING MONITORS ->
RECOVERY USING OSDS, another thing that you lose when you use
ceph-objectstore-tool --op update-mon-db to recover a lost monitor database is
the MDS maps.
That seems like a pretty casual way of saying if your monitor database gets
corrupted, you can kiss your entire cephfs filesystem goodbye. Is that what
it means?
Luckily; it's not. I don't remember if the MDS maps contain entirely ephemeral data, but on the scale of cephfs recovery scenarios that's just about the easiest one. Somebody would have to walk through it; you probably need to look up the table states and mds counts from the RADOS store and generate a new (epoch 1 or 2) mdsmap which contains those settings ready to go. Or maybe you just need to "create" a new cephfs on the prior pools and set it up with the correct number of MDSes.
At the moment the mostly-documented recovery procedure probably involves recovering the journals, flushing everything out, and resetting the server state to a single MDS, and if you lose all your monitors there's a good chance you need to be going through recovery anyway, so...*shrug*
-Greg
Is there a way to recover the MDS maps or otherwise gain access to
all the files once you've recovered access to the OSDs?
--
Bryan Henderson San Jose, California
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
