Hi Greg, Loic, I think we have seen this as well (sent a mail to the list a week or so ago about incomplete pgs). I ended up giving up on the data and doing a force_create_pgs after doing a find on my OSDs and deleting the relevant pg dirs. If there are any logs etc you'd like to see for debugging / post-mortem, I'd be happy to send them along. Cheers, Lincoln On Oct 28, 2014, at 4:11 PM, Gregory Farnum wrote: > On Thu, Oct 23, 2014 at 6:41 AM, Chris Kitzmiller > <ckitzmiller@xxxxxxxxxxxxx> wrote: >> On Oct 22, 2014, at 8:22 PM, Craig Lewis wrote: >> >> Shot in the dark: try manually deep-scrubbing the PG. You could also try >> marking various osd's OUT, in an attempt to get the acting set to include >> osd.25 again, then do the deep-scrub again. That probably won't help >> though, because the pg query says it probed osd.25 already... actually , it >> doesn't. osd.25 is in "probing_osds" not "probed_osds". The deep-scrub >> might move things along. >> >> Re-reading your original post, if you marked the slow osds OUT, but left >> them running, you should not have lost data. >> >> >> That's true. I just marked them out. I did lose osd.10 (in addition to >> out'ting those other two OSDs) so I'm not out of the woods yet. >> >> If the scrubs don't help, it's probably time to hop on IRC. >> >> >> When I issue the deep-scrub command the cluster just doesn't scrub it. Same >> for regular scrub. :( >> >> This pool was offering an RBD which I've lost my connection to and it won't >> remount so my data is totally inaccessible at the moment. Thanks for your >> help so far! > > It looks like you are suffering from > http://tracker.ceph.com/issues/9752, which we've not yet seen in-house > but have had reported a few times. I suspect that Loic (CC'ed) would > like to discuss your cluster's history with you to try and narrow it > down. > -Greg > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |