On 5/27/16, 3:01 PM, "Gregory Farnum" <gfarnum@xxxxxxxxxx> wrote: >> >> So would the next steps be to run the following commands?: >> >> cephfs-table-tool 0 reset session >> cephfs-table-tool 0 reset snap >> cephfs-table-tool 0 reset inode >> cephfs-journal-tool --rank=0 journal reset >> cephfs-data-scan init >> >> cephfs-data-scan scan_extents data >> cephfs-data-scan scan_inodes data > >No, definitely not. I think you just need to reset the journal again, >since you wiped out a bunch of its data with that fs reset command. >Since your backing data should already be consistent you don't need to >do any data scans. Your snap and inode tables might be corrupt, >but...hopefully not. If they are busted...actually, I don't remember; >maybe you will need to run the data scan tooling to repair those. I'd >try to avoid it if possible just because of the time involved. (It'll >become obvious pretty quickly if the inode tables are no good.) So when I attempt to reset the journal again I get this: # cephfs-journal-tool journal reset journal does not exist on-disk. Did you set a bad rank?2016-05-27 15:03:30.016326 7f63f987e700 0 client.20626476.journaler(ro) error getting journal off disk Error loading journal: (2) No such file or directory, pass --force to forcibly reset this journal Error ((2) No such file or directory) And then I tried to force it which seemed to succeed: # cephfs-journal-tool journal reset --force writing EResetJournal entry However, when I restart the mds it gets stuck in standby mode: 2016-05-27 15:05:57.080672 7fe0cccd8700 -1 mds.b4 *** got signal Terminated *** 2016-05-27 15:05:57.080703 7fe0cccd8700 1 mds.b4 suicide. wanted state up:standby 2016-05-27 15:06:04.527203 7f500f28a180 0 set uid:gid to 64045:64045 (ceph:ceph) 2016-05-27 15:06:04.527259 7f500f28a180 0 ceph version 10.2.0 (3a9fba20ec743699b69bd0181dd6c54dc01c64b9), process ceph-mds, pid 19163 2016-05-27 15:06:04.527569 7f500f28a180 0 pidfile_write: ignore empty --pid-file 2016-05-27 15:06:04.637842 7f5008a04700 1 mds.b4 handle_mds_map standby The relevant output from 'ceph -s' looks like this: fsmap e287: 0/1/1 up, 1 up:standby, 1 damaged What am I missing? Thanks, Bryan _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com