Hello, So, I've done the following seps: 1. set noout 2. stop osd2 3. ceph-objectstore-tool remove 4. start osd2 5. repeat step 2-4 on osd 28 and 35 then I've run the ceph pg force_create_pg 3.367. This has left the PG in creating state: # ceph -s cluster 6713d1b8-83da-11e6-aa79-525400d98c5a health HEALTH_WARN 1 pgs stuck inactive 1 pgs stuck unclean 8 requests are blocked > 32 sec noout flag(s) set monmap e3: 3 mons at {tv-dl360-1=10.12.193.73:6789/0,tv-dl360-2=10.12.193.74:6789/0,tv-dl360-3=10.12.193.75:6789/0} election epoch 84, quorum 0,1,2 tv-dl360-1,tv-dl360-2,tv-dl360-3 osdmap e60876: 72 osds: 72 up, 72 in flags noout pgmap v3947626: 4864 pgs, 11 pools, 129 GB data, 22073 objects 423 GB used, 130 TB / 130 TB avail 4863 active+clean 1 creating client io 0 B/s rd, 12846 B/s wr, 2 op/s # ceph health detail HEALTH_WARN 1 pgs stuck inactive; 1 pgs stuck unclean; 8 requests are blocked > 32 sec; 1 osds have slow requests; noout flag(s) set pg 3.367 is stuck inactive since forever, current state creating, last acting [] pg 3.367 is stuck unclean since forever, current state creating, last acting [] 8 ops are blocked > 2097.15 sec 8 ops are blocked > 2097.15 sec on osd.28 1 osds have slow requests noout flag(s) set root@tv-dl360-1:~# As I saw that there is nothing on the osd.2 and the other two are blocked, I have restarted the osd2 service ceph-osd restart id=2 this has unlocked the creation process and the PG became active+clean now when I try to import the dump I get this error: OSD has the store locked. # ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-35/ --journal-path /var/lib/ceph/osd/ceph-35/journal --pgid 3.367 --op import --file pg.3.367.export.OSD.35 OSD has the store locked # How should I proceed further? stop all the three OSDs belonging to this PG, and do the import, and then restart them one by one? Kind regards, Laszlo On 15.03.2017 04:27, Brad Hubbard wrote:
Decide which copy you want to keep and export that with ceph-objectstore-tool Delete all copies on all OSDs with ceph-objectstore-tool (not by deleting the directory on the disk). Use force_create_pg to recreate the pg empty. Use ceph-objectstore-tool to do a rados import on the exported pg copy.
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com