The pgs are not actually inconsistent (that is, I think that all of
the real objects are present and healthy). I think each of those pgs
has one of these duplicate pairs confusing scrub (and also pg removal
-- hence your ENOTEMPTY bug). Once we figure out what's going on,
you'll have to clean them up manually. Do not repair any of these. I
suggest that you simply disable scrub and ignore the inconsistent flag
until we have an idea of what is going on.
-Sam
On Tue, Mar 8, 2016 at 12:06 PM, Jeffrey McDonald <jmcdonal@xxxxxxx> wrote:
> I restarted the OSDs with the 'unfound' objects and now I have none, but I
> have 43 inconsistent PGs that I need to repair.....I only see unfound files
> once issue the 'pg repair'. How do I clear out the inconsistent states?
>
>
> ceph -s
> cluster 5221cc73-869e-4c20-950f-18824ddd6692
> health HEALTH_ERR
> 43 pgs inconsistent
> 3507 scrub errors
> noout flag(s) set
> monmap e9: 3 mons at
> {cephmon1=10.32.16.93:6789/0,cephmon2=10.32.16.85:6789/0,cephmon3=10.32.16.89:6789/0}
> election epoch 112718, quorum 0,1,2 cephmon2,cephmon3,cephmon1
> mdsmap e11408: 1/1/1 up {0=0=up:active}
> osdmap e279630: 449 osds: 449 up, 422 in
> flags noout
> pgmap v26505719: 7788 pgs, 21 pools, 251 TB data, 88784 kobjects
> 412 TB used, 2777 TB / 3190 TB avail
> 7731 active+clean
> 43 active+clean+inconsistent
> 7 active+clean+scrubbing+deep
> 7 active+clean+scrubbing
>
> Jeff
>
> On Tue, Mar 8, 2016 at 2:00 PM, Samuel Just <sjust@xxxxxxxxxx> wrote:
>>
>> Yeah, that procedure should have isolated any filesystem issues. Are
>> there still unfound objects?
>> -sam
>>
>
> --
>
> Jeffrey McDonald, PhD
> Assistant Director for HPC Operations
> Minnesota Supercomputing Institute
> University of Minnesota Twin Cities
> 599 Walter Library email: jeffrey.mcdonald@xxxxxxxxxxx
> 117 Pleasant St SE phone: +1 612 625-6905
> Minneapolis, MN 55455 fax: +1 612 624-8861
>
>
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
