Re: One mds daemon damaged, filesystem is offline. How to recover?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Oh these are filestore OSDs? I didn't expect that.

2021-05-20 23:56:52.223 2227200 0 mds.0.journaler.mdlog(ro) _finish_read got less than expected (1555896) 2021-05-20 23:56:52.223 2229180 0 mds.0.log _replay journaler got error -22, aborting 2021-05-20 23:56:52.223 2229180 -1 log_channel(cluster) log [ERR] : Error loading MDS rank 0: (22) Invalid argument

This confirms that the MDS is not starting because of the object size.
I assume there was some recovery going on when this happened? The OSD uptime from your status was quite short.

I'm not really sure what to do next, I found an old thread [4] with almost the same error, only that the user ended up truncating the objects because the expected size was smaller than the object info, in your case they're the other way around. In any case I would backup all three copies of that object including md5sum before chaning anything.

[4] https://www.spinics.net/lists/ceph-devel/msg16516.html

Zitat von Sagara Wijetunga <sagarawmw@xxxxxxxxx>:

Here are the physical file sizes of the "200.00006048*":
OSD.0:-rw-r--r--  1 ceph  ceph  1540096 May 20 22:47 /var/lib/ceph/osd/ceph-0/current/2.44_head/200.00006048__head_56F5F744__

OSD.1:-rw-r--r--  1 ceph  ceph  1540096 May 20 22:47 /var/lib/ceph/osd/ceph-1/current/2.44_head/DIR_4/DIR_4/200.00006048__head_56F5F744__2

OSD2:-rw-r--r--  1 ceph  ceph  1441792 May 20 22:47 /var/lib/ceph/osd/ceph-2/current/2.44_head/DIR_4/DIR_4/200.00006048__head_56F5F744__2

Sagara

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux