Re: Recover VM Images from Dead Cluster

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Actually these steps didn't work for me - I had an older version of Ceph, so I had to upgrade first.

However, the monmap could be restored, but the OSDs were not found anyway.

But I had success now with the Scripts I mentioned before and I was able to extract all VM images from the raw OSDs without using any Ceph tools. So when having the same issue you can use them to extract your Images.

Thanks for your help!


On 25.12.2016 07:37, Shinobu Kinjo wrote:
On Sun, Dec 25, 2016 at 7:33 AM, Brad Hubbard <bhubbard@xxxxxxxxxx> wrote:
On Sun, Dec 25, 2016 at 3:33 AM, wido@xxxxxxxx <wido@xxxxxxxx> wrote:

Op 24 dec. 2016 om 17:20 heeft L. Bader <ceph-users@xxxxxxxxx> het volgende geschreven:

Do you have any references on this?

I searched for something like this quite a lot and did not find anything...

No, saw it somewhere on the ML I think, but I  am not sure.

I just know it is in development or on a todo somewhere.
Already done I believe.

http://docs.ceph.com/docs/master/rados/troubleshooting/troubleshooting-mon/#recovery-using-osds
That approach should work. But things which you *must* make sure
before taking action are:

  1.You must stop all OSDs which you're going to SSH to before
ceph-objectstore-tool'ing.
  2.You should do that approach *much* more carefully.

So if you have any doubt, please let us know.

HTH.

On 24.12.2016 14:55, wido@xxxxxxxx wrote:

Op 24 dec. 2016 om 14:47 heeft L. Bader <ceph-users@xxxxxxxxx> het volgende geschreven:

Hello,

I have a problem with our (dead) Ceph-Cluster: The configuration seems to be gone (deleted / overwritten) and all monitors are gone aswell. However, we do not have (up-to-date) backups for all VMs (used with Proxmox) and we would like to recover them from "raw" OSDs only (we have all OSDs mounted on one Storage Server). Restoring the cluster itself seems impossible.

Work is on it's way iirc to restore MONs from OSD data.

You might want to search for that, the tracker or Github might help.

To recover the VM images I tried to write a simple tool that:
1) searches all OSDs for udata files
2) Sorts them by Image ID
3) Sorts them by "position" / offset
4) Assembles the 4MB blocks to a single file using dd

(See: https://gitlab.lbader.de/kryptur/ceph-recovery/tree/master )

However, for many (nearly all) images there are missing blocks (empty parts I guess). So I created a 4MB block of Null-Bytes for each missing parts.

The problem is that the created Image is not usable. fdisk detects partitions correctly, but we cannot access the data in any way.

Is there another way to recover the data without having any (working) ceph tools?

Greetings and Merry Christmas :)

Lennart

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


--
Cheers,
Brad
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux