It is still "querying", after 6 days now. I have not tried any
scrubbing options, I'll try them just to see. My next idea was to
clobber osd 8, the one it is supposedly "querying".
I ran into this problem too. I don't know what I did to fix it.
I tried ceph pg scrub <pgid>, ceph pg deep-scrub <pgid>, and ceph
osd scrub <osdid>. None of them had an immediate effect. In the
end, it finally cleared several days later in the middle of the
night. I can't even say what or when it finally cleared. A
different OSDs got kicked out, then rejoined. While everything was
moving from degraded to active+clean, it finally finished probing.
If it's still happening tomorrow, I'd try to find a Geeks on IRC
Duty (http://ceph.com/help/community/).
On 5/3/14 09:43 , Kevin Horan wrote:
> Craig,
> Thanks for your response. I have already marked osd.6 as lost,
> as you suggested. The problem is that it is still querying osd.8
> which is not lost. I don't know why it is stuck there. It has been
> querying osd.8 for 4 days now.
> I also tried deleting the broken RBD image but the operation
> just hangs.
>
> Kevin
>
>
> On 5/1/14 10:11 , kevin horan wrote:
>> Here is how I got into this state. I have only 6 OSDs total,
>> 3 on one host (vashti) and 3 on another (zadok). I set the
>> noout flag so I could reboot zadok. Zadok was down for 2
>> minutes. When it came up ceph began recovering the objects
>> that had not been replicated yet. Before recovery finished,
>> osd.6, on vashti, died (IO errors on disk, whole drive
>> un-recoverable). Since osd.6 had objects that had not yet had
>> a chance to replicate to any OSD on zadok, they were lost. I
>> cannot recover anything further from osd.6.
>>
>>
> I'm pretty far out of my element here, but if osd.6 is gone,
> it might help to mark it lost:
> ceph osd lost 6
>
> I had similiar issues when I lost some PGs. I don't think
> that it actually fixed my issue, but marking osds as lost did
> help Ceph move forward.
>
>
> You could also try deleting the broken RBD image, and see if
> that helps.
>
--
Craig Lewis
Senior Systems Engineer
Office +1.714.602.1309
Email clewis-04jk9TcbgGYP2IHM84UzcNBPR1lH4CV8 at public.gmane.org
<mailto:clewis-04jk9TcbgGYP2IHM84UzcNBPR1lH4CV8 at public.gmane.org>
Central Desktop. Work together in ways you never thought possible.
Connect with us Website <http://www.centraldesktop.com/> | Twitter
<http://www.twitter.com/centraldesktop> | Facebook
<http://www.facebook.com/CentralDesktop> | LinkedIn
<http://www.linkedin.com/groups?gid=147417> | Blog
<http://cdblog.centraldesktop.com/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140507/b06582af/attachment.htm>
