Well not the total solution after all. There is still some metadata and header structure left that I still cannot delete with ceph-objectstore-tool —op remove. It makes a core dump. I think I need to declare the OSD lost anyway to the through this. Unless somebody have a better suggestion? Best, Jesper -------------------------- Jesper Lykkegaard Karlsen Scientific Computing Centre for Structural Biology Department of Molecular Biology and Genetics Aarhus University Universitetsbyen 81 8000 Aarhus C E-mail: jelka@xxxxxxxxx Tlf: +45 50906203 > On 1 Sep 2022, at 22.01, Jesper Lykkegaard Karlsen <jelka@xxxxxxxxx> wrote: > > To answer my own question. > > The removal of the corrupt PG, could be fixed by doing ceph-objectstore-tool fuse mount-thingy. > Then from the mount point, delete everything in the PGs head directory. > > This took only a few seconds (compared to 7.5 days) and after unmount and restart of the OSD it came back online. > > Best, > Jesper > > -------------------------- > Jesper Lykkegaard Karlsen > Scientific Computing > Centre for Structural Biology > Department of Molecular Biology and Genetics > Aarhus University > Universitetsbyen 81 > 8000 Aarhus C > > E-mail: jelka@xxxxxxxxx > Tlf: +45 50906203 > >> On 31 Aug 2022, at 20.53, Jesper Lykkegaard Karlsen <jelka@xxxxxxxxx> wrote: >> >> Hi all, >> >> I wanted to move a PG to an empty OSD, so I could do repairs on it without the whole OSD, which is full of other PG’s, would be effected with extensive downtime. >> >> Thus, I exported the PG with ceph-objectstore-tool, an after successful export I removed it. Unfortunately, the remove command was interrupted midway. >> This resulted in a PG that could not be remove with “ceph-objectstore-tool —op remove ….”, since the header is gone. >> Worse is that the OSD does not boot, due to it can see objects from the removed PG, but cannot access them. >> >> I have tried to remove the individual objects in that PG (also with objectstore-tool), but this process is extremely slow. >> When looping over the >65,000 object, each remove takes ~10 sec and is very compute intensive, which is approximately 7.5 days. >> >> Is the a faster way to get around this? >> >> Mvh. Jesper >> >> -------------------------- >> Jesper Lykkegaard Karlsen >> Scientific Computing >> Centre for Structural Biology >> Department of Molecular Biology and Genetics >> Aarhus University >> Universitetsbyen 81 >> 8000 Aarhus C >> >> E-mail: jelka@xxxxxxxxx >> Tlf: +45 50906203 >> >> _______________________________________________ >> ceph-users mailing list -- ceph-users@xxxxxxx >> To unsubscribe send an email to ceph-users-leave@xxxxxxx > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
