On Thu, Nov 8, 2018 at 3:41 AM Hector Martin <hector@xxxxxxxxxxxxxx> wrote:
I'm experimenting with single-host Ceph use cases, where HA is not
important but data durability is.
How does a Ceph cluster react to its (sole) mon being rolled back to an
earlier state? The idea here is that the mon storage may not be
redundant but would be (atomically, e.g. lvm snapshot and dump) backed
up, say, daily. If the cluster goes down and then is brought back up
with a mon backup that is several days to hours old, while the OSDs are
up to date, what are the potential consequences?
Of course I expect maintenance operations to be affected (obviously any
OSDs added/removed would likely get confused). But what about regular
operation? Things like snapshots and snapshot ranges. Is this likely to
cause data loss, or would the OSDs and clients largely not be affected
as long as the cluster config has not changed?
There's a way of rebuilding the monmap from OSD data:
http://docs.ceph.com/docs/mimic/rados/troubleshooting/troubleshooting-mon/#recovery-using-osds
Would this be preferable to just restoring the mon from a backup?
Yes, do that, don't try and back up your monitor. If you restore a monitor from backup then the monitor — your authoritative data source — will warp back in time on what the OSD peering intervals look like, which snapshots have been deleted and created, etc. It would be a huge disaster and probably every running daemon or client would have to pause IO until the monitor generated enough map epochs to "catch up" — and then the rest of the cluster would start applying those changes and nothing would work right.
What
about the MDS map?
Unlike the OSDMap, the MDSMap doesn't really keep track of any persistent data so it's much safer to rebuild or reset from scratch.
-Greg
--
Hector Martin (hector@xxxxxxxxxxxxxx)
Public Key: https://mrcn.st/pub
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
