On Thu, Aug 27, 2015 at 11:11 AM, John Spray <jspray@xxxxxxxxxx> wrote: > On Thu, Aug 27, 2015 at 9:33 AM, Andrzej Łukawski <alukawski@xxxxxxxxxx> wrote: >> Hi, >> >> I ran cephfs-journal-tool to inspect journal 12 hours ago - it's still >> running. Or... it didn't crush yet, although I don't see any output from it. >> Is it normal behaviour? > > Your cluster is severely damaged, at the RADOS level. I would imagine > that the journal tool is getting stuck the same way the MDS is getting > stuck: when trying to read some objects from a fatally damaged PG, it > is blocking, waiting (in vain) for the PG to become healthy again. Oh, yes, you'll need to persuade RADOS the data isn't coming back first. That generally involves marking OSDs as lost, although I haven't done it recently (ever?). > > It is unlikely that you will get a functioning filesystem back with > existing tools. Even if you could, you would have to fix the > underlying RADOS cluster before doing anything with CephFS. You > really are in "restore from backups" territory here. Indeed. If possible you should just restore. Going through what repair process exists will be tedious, frustrating, and not very successful. But if you don't have backups and want to play around you might get lucky and recover some of the files. -Greg _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |