Re: Troubleshooting Incomplete PGs

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




On 28/10/2014 22:20, Lincoln Bryant wrote:
> Hi Greg, Loic,
> 
> I think we have seen this as well (sent a mail to the list a week or so ago about incomplete pgs). I ended up giving up on the data and doing a force_create_pgs after doing a find on my OSDs and deleting the relevant pg dirs. If there are any logs etc you'd like to see for debugging / post-mortem, I'd be happy to send them along.
> 

Hi,

Feel free to attach to http://tracker.ceph.com/issues/9752 osdmaps and/or links to the relevant threads, that will be very helpful.

Thanks !

> Cheers,
> Lincoln
> 
> On Oct 28, 2014, at 4:11 PM, Gregory Farnum wrote:
> 
>> On Thu, Oct 23, 2014 at 6:41 AM, Chris Kitzmiller
>> <ckitzmiller@xxxxxxxxxxxxx> wrote:
>>> On Oct 22, 2014, at 8:22 PM, Craig Lewis wrote:
>>>
>>> Shot in the dark: try manually deep-scrubbing the PG.  You could also try
>>> marking various osd's OUT, in an attempt to get the acting set to include
>>> osd.25 again, then do the deep-scrub again.  That probably won't help
>>> though, because the pg query says it probed osd.25 already... actually , it
>>> doesn't.  osd.25 is in "probing_osds" not "probed_osds". The deep-scrub
>>> might move things along.
>>>
>>> Re-reading your original post, if you marked the slow osds OUT, but left
>>> them running, you should not have lost data.
>>>
>>>
>>> That's true. I just marked them out. I did lose osd.10 (in addition to
>>> out'ting those other two OSDs) so I'm not out of the woods yet.
>>>
>>> If the scrubs don't help, it's probably time to hop on IRC.
>>>
>>>
>>> When I issue the deep-scrub command the cluster just doesn't scrub it. Same
>>> for regular scrub. :(
>>>
>>> This pool was offering an RBD which I've lost my connection to and it won't
>>> remount so my data is totally inaccessible at the moment. Thanks for your
>>> help so far!
>>
>> It looks like you are suffering from
>> http://tracker.ceph.com/issues/9752, which we've not yet seen in-house
>> but have had reported a few times. I suspect that Loic (CC'ed) would
>> like to discuss your cluster's history with you to try and narrow it
>> down.
>> -Greg
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> 
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> 

-- 
Loïc Dachary, Artisan Logiciel Libre

Attachment: signature.asc
Description: OpenPGP digital signature

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux