On 28/10/2014 22:20, Lincoln Bryant wrote: > Hi Greg, Loic, > > I think we have seen this as well (sent a mail to the list a week or so ago about incomplete pgs). I ended up giving up on the data and doing a force_create_pgs after doing a find on my OSDs and deleting the relevant pg dirs. If there are any logs etc you'd like to see for debugging / post-mortem, I'd be happy to send them along. > Hi, Feel free to attach to http://tracker.ceph.com/issues/9752 osdmaps and/or links to the relevant threads, that will be very helpful. Thanks ! > Cheers, > Lincoln > > On Oct 28, 2014, at 4:11 PM, Gregory Farnum wrote: > >> On Thu, Oct 23, 2014 at 6:41 AM, Chris Kitzmiller >> <ckitzmiller@xxxxxxxxxxxxx> wrote: >>> On Oct 22, 2014, at 8:22 PM, Craig Lewis wrote: >>> >>> Shot in the dark: try manually deep-scrubbing the PG. You could also try >>> marking various osd's OUT, in an attempt to get the acting set to include >>> osd.25 again, then do the deep-scrub again. That probably won't help >>> though, because the pg query says it probed osd.25 already... actually , it >>> doesn't. osd.25 is in "probing_osds" not "probed_osds". The deep-scrub >>> might move things along. >>> >>> Re-reading your original post, if you marked the slow osds OUT, but left >>> them running, you should not have lost data. >>> >>> >>> That's true. I just marked them out. I did lose osd.10 (in addition to >>> out'ting those other two OSDs) so I'm not out of the woods yet. >>> >>> If the scrubs don't help, it's probably time to hop on IRC. >>> >>> >>> When I issue the deep-scrub command the cluster just doesn't scrub it. Same >>> for regular scrub. :( >>> >>> This pool was offering an RBD which I've lost my connection to and it won't >>> remount so my data is totally inaccessible at the moment. Thanks for your >>> help so far! >> >> It looks like you are suffering from >> http://tracker.ceph.com/issues/9752, which we've not yet seen in-house >> but have had reported a few times. I suspect that Loic (CC'ed) would >> like to discuss your cluster's history with you to try and narrow it >> down. >> -Greg >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Loïc Dachary, Artisan Logiciel Libre
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
