Remove Error - "Possible data damage: 2 pgs recovery_unfound"

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Good afternoon!

I have a small Ceph-cluster running with Proxmox, and after an update on one of the nodes and a reboot. So far so good.
But after a couple of hours, I saw this:

root@pve2:~# ceph health detail
HEALTH_ERR 16/1101836 objects unfound (0.001%); Possible data damage: 2 pgs recovery_unfound; Degraded data redundancy: 48/3305508 objects degraded (0.001%), 2 pgs degraded, 2 pgs undersized
OBJECT_UNFOUND 16/1101836 objects unfound (0.001%)
    pg 1.37 has 6 unfound objects
    pg 1.48 has 10 unfound objects
PG_DAMAGED Possible data damage: 2 pgs recovery_unfound
pg 1.37 is active+recovery_unfound+undersized+degraded+remapped, acting [11,17], 6 unfound pg 1.48 is active+recovery_unfound+undersized+degraded+remapped, acting [5,11], 10 unfound PG_DEGRADED Degraded data redundancy: 48/3305508 objects degraded (0.001%), 2 pgs degraded, 2 pgs undersized pg 1.37 is stuck undersized for 446774.454853, current state active+recovery_unfound+undersized+degraded+remapped, last acting [11,17] pg 1.48 is stuck undersized for 446774.459466, current state active+recovery_unfound+undersized+degraded+remapped, last acting [5,11]


root@pve2:~# ceph -s
  cluster:
    id:     76e70c34-bce9-4f86-b049-0054f21c3494
    health: HEALTH_ERR
            16/1101836 objects unfound (0.001%)
            Possible data damage: 2 pgs recovery_unfound
Degraded data redundancy: 48/3305508 objects degraded (0.001%), 2 pgs degraded, 2 pgs undersized

  services:
    mon: 3 daemons, quorum pve3,pve1,pve2 (age 2w)
    mgr: pve3(active, since 2w), standbys: pve1, pve2
    mds: cephfs:1 {0=pve1=up:active} 2 up:standby
    osd: 25 osds: 25 up (since 5d), 25 in (since 8d); 2 remapped pgs

  data:
    pools:   4 pools, 672 pgs
    objects: 1.10M objects, 2.9 TiB
    usage:   8.6 TiB used, 12 TiB / 21 TiB avail
    pgs:     48/3305508 objects degraded (0.001%)
             16/1101836 objects unfound (0.001%)
             669 active+clean
             2   active+recovery_unfound+undersized+degraded+remapped
             1   active+clean+scrubbing+deep

  io:
    client:   680 B/s rd, 2.6 MiB/s wr, 0 op/s rd, 151 op/s wr


I am not really concerned over lost data, since I am 99% sure it belonged to a faulty prometheus server anyway. The question is, how can I remove the warnings without affecting the other objects?

Thankful for any pointers!

--
Jonathan Sélea

Website: https://jonathanselea.se
PGP Key: 0x8B35B3C894B964DD
Fingerprint: 4AF2 10DE 996B 673C 0FD8  AFA0 8B35 B3C8 94B9 64DD
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux