Re: pgs stuck inactive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello,


So, I've done the following seps:

1. set noout
2. stop osd2
3. ceph-objectstore-tool remove
4. start osd2
5. repeat step 2-4 on osd 28 and 35

then I've run the ceph pg force_create_pg 3.367.
This has left the PG in creating state:

# ceph -s
    cluster 6713d1b8-83da-11e6-aa79-525400d98c5a
     health HEALTH_WARN
            1 pgs stuck inactive
            1 pgs stuck unclean
            8 requests are blocked > 32 sec
            noout flag(s) set
     monmap e3: 3 mons at {tv-dl360-1=10.12.193.73:6789/0,tv-dl360-2=10.12.193.74:6789/0,tv-dl360-3=10.12.193.75:6789/0}
            election epoch 84, quorum 0,1,2 tv-dl360-1,tv-dl360-2,tv-dl360-3
     osdmap e60876: 72 osds: 72 up, 72 in
            flags noout
      pgmap v3947626: 4864 pgs, 11 pools, 129 GB data, 22073 objects
            423 GB used, 130 TB / 130 TB avail
                4863 active+clean
                   1 creating
  client io 0 B/s rd, 12846 B/s wr, 2 op/s


# ceph health detail
HEALTH_WARN 1 pgs stuck inactive; 1 pgs stuck unclean; 8 requests are blocked > 32 sec; 1 osds have slow requests; noout flag(s) set
pg 3.367 is stuck inactive since forever, current state creating, last acting []
pg 3.367 is stuck unclean since forever, current state creating, last acting []
8 ops are blocked > 2097.15 sec
8 ops are blocked > 2097.15 sec on osd.28
1 osds have slow requests
noout flag(s) set
root@tv-dl360-1:~#

As I saw that there is nothing on the osd.2 and the other two are blocked, I have restarted the osd2

service ceph-osd restart id=2

this has unlocked the creation process and the PG became active+clean

now when I try to import the dump I get this error: OSD has the store locked.

# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-35/ --journal-path /var/lib/ceph/osd/ceph-35/journal --pgid 3.367 --op import --file pg.3.367.export.OSD.35
OSD has the store locked
#

How should I proceed further?
stop all the three OSDs belonging to this PG, and do the import, and then restart them one by one?

Kind regards,
Laszlo



On 15.03.2017 04:27, Brad Hubbard wrote:
Decide which copy you want to keep and export that with ceph-objectstore-tool

Delete all copies on all OSDs with ceph-objectstore-tool (not by
deleting the directory on the disk).

Use force_create_pg to recreate the pg empty.

Use ceph-objectstore-tool to do a rados import on the exported pg copy.


_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux