Hi, Your cluster was healthy before purging osd? How much time did you wait between stoping osd and purging them? Étienne > On 17 Jan 2022, at 15:24, Rafael Diaz Maurin <Rafael.DiazMaurin@xxxxxxxxxxxxxxx> wrote: > > Hello, > > All my pools on the cluster are replicated (x3). > > I purged some OSD (after I stopped them) and remove the disks from the servers, and now I have 4 PGs in stale+undersized+degraded+peered. > > Reduced data availability: 4 pgs inactive, 4 pgs stale > > pg 1.561 is stuck stale for 39m, current state stale+undersized+degraded+peered, last acting [64] > pg 1.af2 is stuck stale for 39m, current state stale+undersized+degraded+peered, last acting [63] > pg 3.3 is stuck stale for 39m, current state stale+undersized+degraded+peered, last acting [48] > pg 9.5ca is stuck stale for 38m, current state stale+undersized+degraded+peered, last acting [49] > > > Those 4 OSDs have been purged, so ther aren't anymore in the crushmap. > > I tried a pg repair : > ceph pg repair 1.561 > ceph pg repair 1.af2 > ceph pg repair 3.3 > ceph pg repair 9.5ca > > > The PGs are remapped but non of the degraded objects have been repaired > > ceph pg map 9.5ca > osdmap e355782 pg 9.5ca (9.5ca) -> up [54,75,82] acting [54,75,82] > ceph pg map 3.3 > osdmap e355782 pg 3.3 (3.3) -> up [179,180,107] acting [179,180,107] > ceph pg map 1.561 > osdmap e355785 pg 1.561 (1.561) -> up [70,188,87] acting [70,188,87] > ceph pg map 1.af2 > osdmap e355789 pg 1.af2 (1.af2) -> up [189,74,184] acting [189,74,184] > > How can I succeed in reparing my 4 PGs ? > > This affect the cephfs-metadata pool, and the filesystem is degraded because the rank0 mds node stuck in rejoin state. > > > Thank you. > > Rafael > > > -- > Rafael Diaz Maurin > DSI de l'Université de Rennes 1 > Pôle Infrastructures, équipe Systèmes > 02 23 23 71 57 > > > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |