Note that while the "repair" command *will* return your cluster to consistency, it is not guaranteed to restore the data you want to see there ? in general, it will simply put the primary OSD's view of the world on the replicas. If you have a massive inconsistency like that, you probably want to figure out what happened and if it's simply one bad OSD you can remove, or a more general problem. -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Tue, May 27, 2014 at 3:17 PM, Michael <michael at onlinefusion.co.uk> wrote: > Hi Peter, > > Please use "ceph pg repair XX.xx". It might take a few seconds to kick in > after being instructed. > > -Michael > > > On 27/05/2014 21:40, phowell wrote: >> >> Hi >> >> First apologies if this is the wrong place to ask this question. >> >> We are running a small Ceph (0.79) cluster will about 12 osd's which are >> on top of a zfs raid 1+0 (for another discussion)... which were created on >> this version. >> >> Below I have included the ceph health detail which has rather a lot of >> inconsistent placement groups. The underlying zfs arrays have reported no >> problems and all the zpools are good. None of the 12 osd's are down or out. >> >> I could not find a way to repair these placement groups. I have tried >> using 'ceph pg scrub XX.xx' with no effect. I would be grateful if someone >> could point me in the right direction to fix the problem. >> >> Thanks >> >> Peter Howell. >> >> root at durotar:/var/log/ceph# ceph health detail >> HEALTH_ERR 86 pgs inconsistent; 292 scrub errors >> pg 42.57 is active+clean+inconsistent, acting [2,3,10] >> pg 42.56 is active+clean+inconsistent, acting [8,9,1] >> pg 42.50 is active+clean+inconsistent, acting [4,1,9] >> pg 42.53 is active+clean+inconsistent, acting [3,1,8] >> pg 42.52 is active+clean+inconsistent, acting [11,7,3] >> pg 42.5d is active+clean+inconsistent, acting [4,1,6] >> pg 42.5c is active+clean+inconsistent, acting [7,9,5] >> pg 42.5e is active+clean+inconsistent, acting [6,10,0] >> pg 42.59 is active+clean+inconsistent, acting [5,0,11] >> pg 42.5b is active+clean+inconsistent, acting [1,3,6] >> pg 42.5a is active+clean+inconsistent, acting [8,9,4] >> pg 42.45 is active+clean+inconsistent, acting [3,0,9] >> pg 42.44 is active+clean+inconsistent, acting [10,8,5] >> pg 42.46 is active+clean+inconsistent, acting [5,0,6] >> pg 42.41 is active+clean+inconsistent, acting [5,2,8] >> pg 42.42 is active+clean+inconsistent, acting [7,10,5] >> pg 42.4d is active+clean+inconsistent, acting [1,5,11] >> pg 42.4c is active+clean+inconsistent, acting [0,5,6] >> pg 42.4f is active+clean+inconsistent, acting [6,9,1] >> pg 42.4e is active+clean+inconsistent, acting [5,0,6] >> pg 42.49 is active+clean+inconsistent, acting [6,9,0] >> pg 42.48 is active+clean+inconsistent, acting [2,4,7] >> pg 42.4a is active+clean+inconsistent, acting [9,8,4] >> pg 42.74 is active+clean+inconsistent, acting [3,2,11] >> pg 42.77 is active+clean+inconsistent, acting [5,2,8] >> pg 42.76 is active+clean+inconsistent, acting [9,7,3] >> pg 42.71 is active+clean+inconsistent, acting [8,10,0] >> pg 42.7c is active+clean+inconsistent, acting [8,10,0] >> pg 42.7f is active+clean+inconsistent, acting [10,6,0] >> pg 42.7e is active+clean+inconsistent, acting [2,4,9] >> pg 42.78 is active+clean+inconsistent, acting [4,1,11] >> pg 42.65 is active+clean+inconsistent, acting [8,11,4] >> pg 42.67 is active+clean+inconsistent, acting [0,3,11] >> pg 42.66 is active+clean+inconsistent, acting [4,1,7] >> pg 42.6c is active+clean+inconsistent, acting [4,2,6] >> pg 42.6f is active+clean+inconsistent, acting [9,7,5] >> pg 42.6e is active+clean+inconsistent, acting [8,11,4] >> pg 42.69 is active+clean+inconsistent, acting [10,8,4] >> pg 42.68 is active+clean+inconsistent, acting [9,7,0] >> pg 42.6b is active+clean+inconsistent, acting [10,7,4] >> pg 42.6a is active+clean+inconsistent, acting [10,8,2] >> pg 42.15 is active+clean+inconsistent, acting [9,6,4] >> pg 42.14 is active+clean+inconsistent, acting [6,10,3] >> pg 42.16 is active+clean+inconsistent, acting [4,2,9] >> pg 42.11 is active+clean+inconsistent, acting [6,9,2] >> pg 42.10 is active+clean+inconsistent, acting [11,6,2] >> pg 42.1f is active+clean+inconsistent, acting [4,0,10] >> pg 42.1e is active+clean+inconsistent, acting [10,7,1] >> pg 42.19 is active+clean+inconsistent, acting [3,0,9] >> pg 42.18 is active+clean+inconsistent, acting [5,0,9] >> pg 42.1b is active+clean+inconsistent, acting [7,9,5] >> pg 42.1a is active+clean+inconsistent, acting [3,0,9] >> pg 42.5 is active+clean+inconsistent, acting [11,7,3] >> pg 42.7 is active+clean+inconsistent, acting [7,11,0] >> pg 42.0 is active+clean+inconsistent, acting [7,10,5] >> pg 42.2 is active+clean+inconsistent, acting [11,6,4] >> pg 42.c is active+clean+inconsistent, acting [7,9,5] >> pg 42.f is active+clean+inconsistent, acting [2,3,7] >> pg 42.9 is active+clean+inconsistent, acting [2,3,10] >> pg 42.8 is active+clean+inconsistent, acting [10,7,2] >> pg 42.a is active+clean+inconsistent, acting [9,7,0] >> pg 42.35 is active+clean+inconsistent, acting [9,7,1] >> pg 42.37 is active+clean+inconsistent, acting [2,4,9] >> pg 42.36 is active+clean+inconsistent, acting [3,1,6] >> pg 42.30 is active+clean+inconsistent, acting [4,1,8] >> pg 42.33 is active+clean+inconsistent, acting [3,1,10] >> pg 42.32 is active+clean+inconsistent, acting [1,4,9] >> pg 42.3d is active+clean+inconsistent, acting [1,3,8] >> pg 42.3c is active+clean+inconsistent, acting [8,10,5] >> pg 42.3e is active+clean+inconsistent, acting [2,3,10] >> pg 42.39 is active+clean+inconsistent, acting [1,5,6] >> pg 42.3b is active+clean+inconsistent, acting [5,0,7] >> pg 42.3a is active+clean+inconsistent, acting [7,10,1] >> pg 42.25 is active+clean+inconsistent, acting [3,1,7] >> pg 42.24 is active+clean+inconsistent, acting [0,5,11] >> pg 42.26 is active+clean+inconsistent, acting [0,4,9] >> pg 42.21 is active+clean+inconsistent, acting [8,10,4] >> pg 42.20 is active+clean+inconsistent, acting [9,6,0] >> pg 42.23 is active+clean+inconsistent, acting [0,5,10] >> pg 42.22 is active+clean+inconsistent, acting [1,5,8] >> pg 42.2d is active+clean+inconsistent, acting [11,8,5] >> pg 42.2c is active+clean+inconsistent, acting [5,1,6] >> pg 42.2f is active+clean+inconsistent, acting [1,5,8] >> pg 42.29 is active+clean+inconsistent, acting [5,2,11] >> pg 42.28 is active+clean+inconsistent, acting [1,5,11] >> pg 42.2a is active+clean+inconsistent, acting [11,8,4] >> 292 scrub errors >> >> --- >> >> Follow us on: >> www.twitter.com/teamenergyeaa >> www.youtube.com/user/teamenergyeaa >> >> Subscribe to our newsletter >> http://www.teamenergy.com/newsletter-subscription/ >> www.teamenergy.com +44 (0)1908 690018 enquiries at teamenergy.com >> Registered Office: TEAM (Energy Auditing Agency Ltd), 34 The Forum, >> Rockingham Drive, Linford Wood, Milton Keynes, MK14 6LY >> Registered in England No. 1916768 >> _______________________________________________ >> ceph-users mailing list >> ceph-users at lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > _______________________________________________ > ceph-users mailing list > ceph-users at lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |