Root affected got more than 70TB free. The only solution is manual reweight the OSD. But in this situacion balancer in unmap mode should move data to get all HEALTHY Hope some fix come in the next 14.2.X to fix that issue. Ceph 14.2.2 Centos 7.6 cluster: id: e1ee8086-7cce-43fd-a252-3d677af22428 health: HEALTH_ERR 2 nearfull osd(s) 2 pool(s) nearfull Degraded data redundancy (low space): 9 pgs backfill_toofull services: mon: 3 daemons, quorum CEPH-MON01,CEPH002,CEPH003 (age 19m) mgr: CEPH001(active, since 12h), standbys: CEPH-MON01 osd: 90 osds: 90 up (since 4h), 90 in (since 34h); 9 remapped pgs rgw: 1 daemon active (ceph-rgw03) data: pools: 18 pools, 8252 pgs objects: 105.59M objects, 294 TiB usage: 340 TiB used, 84 TiB / 424 TiB avail pgs: 197930/116420259 objects misplaced (0.170%) 8243 active+clean 9 active+remapped+backfill_toofull ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS TYPE NAME -41 392.89362 - 393 TiB 317 TiB 317 TiB 19 MiB 561 GiB 75 TiB 0 0 - root archive -44 130.95793 - 131 TiB 106 TiB 106 TiB 608 KiB 188 GiB 25 TiB 81.29 1.01 - host CEPH005 83 archive 10.91309 1.00000 11 TiB 9.2 TiB 9.1 TiB 56 KiB 16 GiB 1.7 TiB 83.97 1.05 131 up osd.83 84 archive 10.91309 1.00000 11 TiB 8.5 TiB 8.5 TiB 56 KiB 15 GiB 2.4 TiB 78.33 0.98 121 up osd.84 85 archive 10.91309 1.00000 11 TiB 9.0 TiB 9.0 TiB 92 KiB 16 GiB 1.9 TiB 82.24 1.03 129 up osd.85 86 archive 10.91309 1.00000 11 TiB 10 TiB 10 TiB 48 KiB 18 GiB 597 GiB 94.66 1.18 148 up osd.86 87 archive 10.91399 1.00000 11 TiB 10 TiB 10 TiB 88 KiB 18 GiB 596 GiB 94.66 1.18 149 up osd.87 88 archive 10.91309 1.00000 11 TiB 8.3 TiB 8.3 TiB 56 KiB 14 GiB 2.6 TiB 76.02 0.95 119 up osd.88 97 archive 10.91309 1.00000 11 TiB 7.7 TiB 7.7 TiB 44 KiB 14 GiB 3.2 TiB 70.44 0.88 109 up osd.97 98 archive 10.91309 1.00000 11 TiB 8.4 TiB 8.4 TiB 72 KiB 15 GiB 2.5 TiB 77.18 0.96 126 up osd.98 99 archive 10.91309 1.00000 11 TiB 8.0 TiB 8.0 TiB 24 KiB 15 GiB 2.9 TiB 73.29 0.91 116 up osd.99 100 archive 10.91309 1.00000 11 TiB 9.5 TiB 9.4 TiB 32 KiB 17 GiB 1.4 TiB 86.72 1.08 132 up osd.100 101 archive 10.91309 1.00000 11 TiB 8.4 TiB 8.4 TiB 12 KiB 15 GiB 2.5 TiB 76.71 0.96 125 up osd.101 102 archive 10.91309 1.00000 11 TiB 8.9 TiB 8.8 TiB 28 KiB 15 GiB 2.1 TiB 81.20 1.01 125 up osd.102 -17 0 - 0 B 0 B 0 B 0 B 0 B 0 B 0 0 - host CEPH006 -26 130.96783 - 131 TiB 104 TiB 104 TiB 6.7 MiB 184 GiB 27 TiB 79.21 0.99 - host CEPH007 14 archive 10.91399 1.00000 11 TiB 8.3 TiB 8.3 TiB 28 KiB 15 GiB 2.6 TiB 76.06 0.95 126 up osd.14 15 archive 10.91399 1.00000 11 TiB 8.9 TiB 8.9 TiB 84 KiB 16 GiB 2.0 TiB 81.72 1.02 130 up osd.15 16 archive 10.91399 1.00000 11 TiB 8.7 TiB 8.7 TiB 80 KiB 15 GiB 2.2 TiB 79.98 1.00 127 up osd.16 39 archive 10.91399 1.00000 11 TiB 8.1 TiB 8.1 TiB 3.4 MiB 14 GiB 2.8 TiB 74.26 0.93 118 up osd.39 40 archive 10.91399 1.00000 11 TiB 9.2 TiB 9.2 TiB 53 KiB 16 GiB 1.7 TiB 84.53 1.05 132 up osd.40 44 archive 10.91399 1.00000 11 TiB 8.1 TiB 8.1 TiB 2.6 MiB 15 GiB 2.8 TiB 74.40 0.93 117 up osd.44 48 archive 10.91399 1.00000 11 TiB 9.7 TiB 9.7 TiB 44 KiB 17 GiB 1.2 TiB 89.02 1.11 135 up osd.48 49 archive 10.91399 1.00000 11 TiB 8.6 TiB 8.6 TiB 132 KiB 15 GiB 2.3 TiB 78.90 0.98 126 up osd.49 52 archive 10.91399 1.00000 11 TiB 9.3 TiB 9.3 TiB 28 KiB 17 GiB 1.6 TiB 85.28 1.06 134 up osd.52 77 archive 10.91399 1.00000 11 TiB 8.1 TiB 8.1 TiB 73 KiB 15 GiB 2.8 TiB 74.44 0.93 118 up osd.77 89 archive 10.91399 1.00000 11 TiB 7.2 TiB 7.2 TiB 60 KiB 13 GiB 3.7 TiB 66.22 0.83 106 up osd.89 90 archive 10.91399 1.00000 11 TiB 9.4 TiB 9.3 TiB 48 KiB 16 GiB 1.6 TiB 85.68 1.07 137 up osd.90 -31 130.96783 - 131 TiB 107 TiB 107 TiB 12 MiB 189 GiB 24 TiB 81.86 1.02 - host CEPH008 5 archive 10.91399 1.00000 11 TiB 9.6 TiB 9.6 TiB 2.7 MiB 17 GiB 1.3 TiB 87.81 1.10 135 up osd.5 6 archive 10.91399 1.00000 11 TiB 8.4 TiB 8.4 TiB 3.9 MiB 16 GiB 2.5 TiB 77.19 0.96 124 up osd.6 11 archive 10.91399 1.00000 11 TiB 8.9 TiB 8.8 TiB 48 KiB 16 GiB 2.1 TiB 81.11 1.01 128 up osd.11 45 archive 10.91399 1.00000 11 TiB 9.5 TiB 9.4 TiB 48 KiB 17 GiB 1.5 TiB 86.66 1.08 138 up osd.45 46 archive 10.91399 1.00000 11 TiB 8.9 TiB 8.8 TiB 4.8 MiB 16 GiB 2.1 TiB 81.21 1.01 126 up osd.46 47 archive 10.91399 1.00000 11 TiB 8.4 TiB 8.4 TiB 32 KiB 15 GiB 2.5 TiB 77.22 0.96 118 up osd.47 55 archive 10.91399 1.00000 11 TiB 9.0 TiB 9.0 TiB 28 KiB 16 GiB 1.9 TiB 82.33 1.03 125 up osd.55 70 archive 10.91399 1.00000 11 TiB 9.0 TiB 9.0 TiB 80 KiB 16 GiB 1.9 TiB 82.20 1.03 132 up osd.70 71 archive 10.91399 1.00000 11 TiB 8.2 TiB 8.2 TiB 32 KiB 15 GiB 2.7 TiB 75.47 0.94 118 up osd.71 78 archive 10.91399 1.00000 11 TiB 8.8 TiB 8.8 TiB 61 KiB 15 GiB 2.1 TiB 80.40 1.00 119 up osd.78 79 archive 10.91399 1.00000 11 TiB 9.7 TiB 9.7 TiB 49 KiB 17 GiB 1.2 TiB 89.03 1.11 134 up osd.79 91 archive 10.91399 1.00000 11 TiB 8.9 TiB 8.9 TiB 24 KiB 16 GiB 2.0 TiB 81.69 1.02 130 up osd.91 Regards, Manuel |
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
