Re: Repair inconsistent pgs..

[Irek Fasikhov <malmyzh@xxxxxxxxx>]

Hi, Igor.

You need to repair the PG.

for i in `ceph pg dump| grep inconsistent | grep -v 'inconsistent+repair' | awk {'print$1'}`;do ceph pg repair $i;done

С уважением, Фасихов Ирек Нургаязович

Моб.: +79229045757

2015-08-18 8:27 GMT+03:00 Voloshanenko Igor <igor.voloshanenko@xxxxxxxxx>:

Hi all, at our production cluster, due high rebalancing ((( we have 2 pgs in inconsistent state...

root@temp:~# ceph health detail | grep inc

HEALTH_ERR 2 pgs inconsistent; 18 scrub errors

pg 2.490 is active+clean+inconsistent, acting [56,15,29]

pg 2.c4 is active+clean+inconsistent, acting [56,10,42]

From OSD logs, after recovery attempt:

root@test:~# ceph pg dump | grep -i incons | cut -f 1 | while read i; do ceph pg repair ${i} ; done

dumped all in format plain

instructing pg 2.490 on osd.56 to repair

instructing pg 2.c4 on osd.56 to repair

/var/log/ceph/ceph-osd.56.log:51:2015-08-18 07:26:37.035910 7f94663b3700 -1 log_channel(cluster) log [ERR] : deep-scrub 2.490 f5759490/rbd_data.1631755377d7e.00000000000004da/head//2 expected clone 90c59490/rbd_data.eb486436f2beb.0000000000007a65/141//2

/var/log/ceph/ceph-osd.56.log:52:2015-08-18 07:26:37.035960 7f94663b3700 -1 log_channel(cluster) log [ERR] : deep-scrub 2.490 fee49490/rbd_data.12483d3ba0794b.000000000000522f/head//2 expected clone f5759490/rbd_data.1631755377d7e.00000000000004da/141//2

/var/log/ceph/ceph-osd.56.log:53:2015-08-18 07:26:37.036133 7f94663b3700 -1 log_channel(cluster) log [ERR] : deep-scrub 2.490 a9b39490/rbd_data.12483d3ba0794b.00000000000037b3/head//2 expected clone fee49490/rbd_data.12483d3ba0794b.000000000000522f/141//2

/var/log/ceph/ceph-osd.56.log:54:2015-08-18 07:26:37.036243 7f94663b3700 -1 log_channel(cluster) log [ERR] : deep-scrub 2.490 bac19490/rbd_data.1238e82ae8944a.000000000000032e/head//2 expected clone a9b39490/rbd_data.12483d3ba0794b.00000000000037b3/141//2

/var/log/ceph/ceph-osd.56.log:55:2015-08-18 07:26:37.036289 7f94663b3700 -1 log_channel(cluster) log [ERR] : deep-scrub 2.490 98519490/rbd_data.123e9c2ae8944a.0000000000000807/head//2 expected clone bac19490/rbd_data.1238e82ae8944a.000000000000032e/141//2

/var/log/ceph/ceph-osd.56.log:56:2015-08-18 07:26:37.036314 7f94663b3700 -1 log_channel(cluster) log [ERR] : deep-scrub 2.490 c3c09490/rbd_data.1238e82ae8944a.0000000000000c2b/head//2 expected clone 98519490/rbd_data.123e9c2ae8944a.0000000000000807/141//2

/var/log/ceph/ceph-osd.56.log:57:2015-08-18 07:26:37.036363 7f94663b3700 -1 log_channel(cluster) log [ERR] : deep-scrub 2.490 28809490/rbd_data.edea7460fe42b.00000000000001d9/head//2 expected clone c3c09490/rbd_data.1238e82ae8944a.0000000000000c2b/141//2

/var/log/ceph/ceph-osd.56.log:58:2015-08-18 07:26:37.036432 7f94663b3700 -1 log_channel(cluster) log [ERR] : deep-scrub 2.490 e1509490/rbd_data.1423897545e146.00000000000009a6/head//2 expected clone 28809490/rbd_data.edea7460fe42b.00000000000001d9/141//2

/var/log/ceph/ceph-osd.56.log:59:2015-08-18 07:26:38.548765 7f94663b3700 -1 log_channel(cluster) log [ERR] : 2.490 deep-scrub 17 errors

So, how i can solve "expected clone" situation by hand? 

Thank in advance!

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com