Re: Inconsistent PG / Impossible deep-scrub

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Finally, after having corruption with the MDS I had no choice but to try to manually repair the PG.

Following the procedure on the blog post from Ceph at http://ceph.com/planet/ceph-manually-repair-object/ I was able to get the PG back to active+clean, ceph pg repair wasn't still not working and an automatic deep-scrub confirmed the object was still faulty.

On Fri, Dec 18, 2015 at 10:42 AM, Jérôme Poulin <jeromepoulin@xxxxxxxxx> wrote:
Good day everyone,

I currently manage a Ceph cluster running Firefly 0.80.10, we had some maintenance which implied stopping OSD and starting them back again. This caused one of the hard drive to notice it had a bad sector and then Ceph to mark it as inconsistent.

After reparing the physical issue, I went and tried ceph pg repair, no action, then I tried ceph pg deep-scrub, still no action.

I verified the log of each OSD which had the PG and confirmed that nothing was logged, no repair, no deep-scrub. After trying deep-scrubbing manually other PGs, I confirmed that my requests were completely ignored.

The only flag set is noout since this cluster is too small, but automatic deep-scrubs are working and are logged both in ceph.log and the OSD log.

I tried restarting the monitor in charge to elect a new one and restart each affected OSD for the inconsistent PG with no success.

I also tried to fix the defective object myself in case it was hanging something, now the object has the same checksum on each OSD.

Is there a way to ask the OSD directly to deep-scrub without using the monitor? Is there a known issue about commands getting ignored?

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux