ceph (jewel) unable to recover after node failure

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



here is the output of ceph health detail: 

HEALTH_ERR 16 pgs are stuck inactive for more than 300 seconds; 134 pgs backfill_wait; 11 pgs backfilling; 69 pgs degraded; 14 pgs down; 2 pgs incomplete; 14 pgs peering; 6 pgs recovery_wait; 69 pgs stuck degraded; 16 pgs stuck inactive; 167 pgs stuck unclean; 63 pgs stuck undersized; 63 pgs undersized; 29 requests are blocked > 32 sec; 6 osds have slow requests; recovery 667605/71152293 objects degraded (0.938%); recovery 1564114/71152293 objects misplaced (2.198%); too many PGs per OSD (314 > max 300)
pg 8.3ec is stuck inactive for 17320.100016, current state down+peering, last acting [22,40,49]
pg 9.3ac is stuck inactive since forever, current state down+remapped+peering, last acting [36]
pg 9.243 is stuck inactive for 17602.030517, current state incomplete, last acting [34,47,26]
pg 9.23e is stuck inactive since forever, current state down+remapped+peering, last acting [18]
pg 11.7a is stuck inactive since forever, current state down+remapped+peering, last acting [13,25]
pg 9.66 is stuck inactive since forever, current state down+remapped+peering, last acting [20]
pg 8.6c is stuck inactive for 17196.609471, current state down+peering, last acting [34,17,48]
pg 8.143 is stuck inactive for 17201.229429, current state down+remapped+peering, last acting [39,19]
pg 10.103 is stuck inactive for 17544.862477, current state down+peering, last acting [30,19,53]
pg 8.ae is stuck inactive for 17518.839339, current state down+peering, last acting [39,21,52]
pg 8.37 is stuck inactive for 17520.793755, current state down+peering, last acting [15,40,52]
pg 7.399 is stuck inactive since forever, current state down+remapped+peering, last acting [21]
pg 7.210 is stuck inactive for 17535.412721, current state incomplete, last acting [22,49,15]
pg 7.136 is stuck inactive for 40796.009480, current state down+remapped+peering, last acting [46]
pg 9.38 is stuck inactive since forever, current state down+remapped+peering, last acting [46]
pg 7.36 is stuck inactive since forever, current state down+remapped+peering, last acting [20]
pg 9.3ff is stuck unclean for 59505.890789, current state active+remapped+wait_backfill, last acting [48,53,33]
pg 9.3e8 is stuck unclean for 21312.446345, current state active+remapped+wait_backfill, last acting [28,53,27]
pg 9.3df is stuck unclean for 17346.719500, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,46]
pg 7.3c8 is stuck unclean for 86528.672542, current state active+remapped+wait_backfill, last acting [30,35,40]
pg 9.3b1 is stuck unclean for 17859.207821, current state active+remapped+wait_backfill, last acting [35,40,14]
pg 7.3b8 is stuck unclean for 88517.511151, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,14]
pg 9.398 is stuck unclean for 41016.001863, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,12]
pg 7.38b is stuck unclean for 41003.853238, current state active+remapped+wait_backfill, last acting [13,34,42]
pg 7.36d is stuck unclean for 18780.388726, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,29]
pg 9.363 is stuck unclean for 59589.647646, current state active+remapped+wait_backfill, last acting [40,16,32]
pg 7.369 is stuck unclean for 17601.998787, current state active+undersized+degraded+remapped+wait_backfill, last acting [31,15]
pg 9.368 is stuck unclean for 41558.892612, current state active+remapped+wait_backfill, last acting [21,25,19]
pg 7.34d is stuck unclean for 41015.946070, current state active+remapped+wait_backfill, last acting [48,14,22]
pg 9.3db is stuck unclean for 50487.572088, current state active+remapped+wait_backfill, last acting [40,33,52]
pg 7.30c is stuck unclean for 98943.868376, current state active+remapped+wait_backfill, last acting [12,39,16]
pg 7.3a5 is stuck unclean for 26487.349029, current state active+remapped+wait_backfill, last acting [36,28,33]
pg 8.2d3 is stuck unclean for 98535.669203, current state active+recovery_wait+degraded, last acting [30,33,52]
pg 7.2d6 is stuck unclean for 17769.739311, current state active+remapped+wait_backfill, last acting [16,15,36]
pg 9.2b2 is stuck unclean for 67277.008904, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,19]
pg 9.2b5 is stuck unclean for 17510.383905, current state active+remapped+wait_backfill, last acting [32,29,33]
pg 9.2b8 is stuck unclean for 17601.978526, current state active+remapped+backfilling, last acting [18,21,50]
pg 9.2a1 is stuck unclean for 41018.243699, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,49]
pg 9.2a8 is stuck unclean for 59129.277638, current state active+remapped+wait_backfill, last acting [15,17,44]
pg 7.295 is stuck unclean for 17859.207323, current state active+undersized+degraded+remapped+wait_backfill, last acting [38,21]
pg 7.28b is stuck unclean for 41058.653702, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,14]
pg 9.28c is stuck unclean for 64378.896968, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,15]
pg 7.27a is stuck unclean for 26487.377599, current state active+undersized+degraded+remapped+backfilling, last acting [31,34]
pg 7.272 is stuck unclean for 17601.979713, current state active+remapped+wait_backfill, last acting [20,15,36]
pg 7.25d is stuck unclean for 27187.590428, current state active+remapped+wait_backfill, last acting [32,20,31]
pg 7.255 is stuck unclean for 41015.997867, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,36]
pg 9.243 is stuck unclean for 17602.031230, current state incomplete, last acting [34,47,26]
pg 9.24b is stuck unclean for 17519.381051, current state active+remapped+wait_backfill, last acting [16,21,32]
pg 9.24e is stuck unclean for 17859.230523, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,17]
pg 7.235 is stuck unclean for 86354.470989, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,15]
pg 9.23b is stuck unclean for 28119.240226, current state active+remapped+wait_backfill, last acting [12,18,38]
pg 9.23e is stuck unclean since forever, current state down+remapped+peering, last acting [18]
pg 9.3a7 is stuck unclean for 59177.659209, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,30]
pg 7.227 is stuck unclean for 28907.901485, current state active+remapped+wait_backfill, last acting [40,52,45]
pg 7.399 is stuck unclean since forever, current state down+remapped+peering, last acting [21]
pg 7.217 is stuck unclean for 41011.987041, current state active+remapped+wait_backfill, last acting [13,21,47]
pg 9.3ac is stuck unclean since forever, current state down+remapped+peering, last acting [36]
pg 7.2d9 is stuck unclean for 27104.049046, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,38]
pg 7.210 is stuck unclean for 17859.274157, current state incomplete, last acting [22,49,15]
pg 7.20e is stuck unclean for 86626.368778, current state active+remapped+wait_backfill, last acting [31,54,41]
pg 9.393 is stuck unclean for 64329.134889, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,45]
pg 7.79 is stuck unclean for 19806.618967, current state active+remapped+wait_backfill, last acting [21,22,14]
pg 8.1b8 is stuck unclean for 41014.414517, current state active+recovery_wait+degraded, last acting [40,49,28]
pg 8.147 is stuck unclean for 164369.124456, current state active+recovery_wait+degraded, last acting [30,52,16]
pg 9.1c1 is stuck unclean for 73541.363717, current state active+undersized+degraded+remapped+backfilling, last acting [33,23]
pg 7.43 is stuck unclean for 17570.722072, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,27]
pg 9.34c is stuck unclean for 17858.967832, current state active+remapped+wait_backfill, last acting [21,18,40]
pg 7.2e is stuck unclean for 17006.897559, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,16]
pg 9.e2 is stuck unclean for 73546.869124, current state active+remapped+backfilling, last acting [12,32,41]
pg 7.ec is stuck unclean for 86623.719450, current state active+remapped+wait_backfill, last acting [38,16,42]
pg 9.1cf is stuck unclean for 17859.163275, current state active+remapped+wait_backfill, last acting [23,24,37]
pg 9.16c is stuck unclean for 17601.975335, current state active+remapped+backfilling, last acting [14,52,51]
pg 7.3a7 is stuck unclean for 88230.251484, current state active+remapped+wait_backfill, last acting [38,24,37]
pg 9.142 is stuck unclean for 17600.748012, current state active+remapped+backfilling, last acting [48,40,46]
pg 8.143 is stuck unclean for 17602.012378, current state down+remapped+peering, last acting [39,19]
pg 9.e3 is stuck unclean for 64255.879860, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,37]
pg 7.36 is stuck unclean since forever, current state down+remapped+peering, last acting [20]
pg 9.38 is stuck unclean since forever, current state down+remapped+peering, last acting [46]
pg 9.36f is stuck unclean for 17011.080538, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,48]
pg 7.1df is stuck unclean for 23673.901192, current state active+undersized+degraded+remapped+wait_backfill, last acting [38,36]
pg 9.1d0 is stuck unclean for 64372.318399, current state active+remapped+wait_backfill, last acting [19,33,17]
pg 9.36d is stuck unclean for 17602.039894, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,21]
pg 7.1d1 is stuck unclean for 41018.200731, current state active+remapped+wait_backfill, last acting [26,52,37]
pg 9.58 is stuck unclean for 59128.167782, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,18]
pg 9.22c is stuck unclean for 17859.207622, current state active+remapped+wait_backfill, last acting [17,52,23]
pg 9.157 is stuck unclean for 17601.999929, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,42]
pg 7.1ec is stuck unclean for 24759.249961, current state active+undersized+degraded+remapped+wait_backfill, last acting [51,30]
pg 8.e is stuck unclean for 17601.973482, current state active+recovery_wait+degraded, last acting [30,43,52]
pg 8.3ec is stuck unclean for 17602.074400, current state down+peering, last acting [22,40,49]
pg 9.c1 is stuck unclean for 28853.325800, current state active+remapped+wait_backfill, last acting [31,34,30]
pg 7.1ef is stuck unclean for 17837.914301, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,52]
pg 7.1cb is stuck unclean for 41018.214477, current state active+remapped+wait_backfill, last acting [20,47,32]
pg 9.218 is stuck unclean for 59128.468037, current state active+remapped+wait_backfill, last acting [35,29,41]
pg 9.8a is stuck unclean for 41382.816787, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,44]
pg 9.33d is stuck unclean for 64408.309381, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,20]
pg 9.1af is stuck unclean for 59128.685913, current state active+remapped+wait_backfill, last acting [17,45,21]
pg 7.147 is stuck unclean for 17525.805171, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,21]
pg 9.20e is stuck unclean for 17769.612461, current state active+remapped+wait_backfill, last acting [12,36,51]
pg 11.7a is stuck unclean since forever, current state down+remapped+peering, last acting [13,25]
pg 7.111 is stuck unclean for 65334.462417, current state active+remapped+wait_backfill, last acting [23,45,29]
pg 9.46 is stuck unclean for 17006.518756, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,16]
pg 9.b9 is stuck unclean for 73546.487183, current state active+remapped+wait_backfill, last acting [13,18,43]
pg 7.b7 is stuck unclean for 88535.361984, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,18]
pg 9.352 is stuck unclean for 19770.927405, current state active+remapped+wait_backfill, last acting [36,54,30]
pg 8.37 is stuck unclean for 17601.979356, current state down+peering, last acting [15,40,52]
pg 9.67 is stuck unclean for 41015.976179, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,42]
pg 7.387 is stuck unclean for 17567.700109, current state active+undersized+degraded+remapped+wait_backfill, last acting [21,29]
pg 9.2b0 is stuck unclean for 70684.124685, current state active+remapped+wait_backfill, last acting [52,11,21]
pg 7.63 is stuck unclean for 97857.910817, current state active+undersized+degraded+remapped+backfilling, last acting [27,15]
pg 8.6c is stuck unclean for 17602.028215, current state down+peering, last acting [34,17,48]
pg 7.1f5 is stuck unclean for 17839.976322, current state active+remapped+wait_backfill, last acting [23,27,17]
pg 7.2b9 is stuck unclean for 46402.960930, current state active+remapped+wait_backfill, last acting [32,47,50]
pg 9.2b7 is stuck unclean for 28834.004113, current state active+remapped+wait_backfill, last acting [23,51,38]
pg 7.6e is stuck unclean for 88230.962468, current state active+remapped+wait_backfill, last acting [13,16,43]
pg 7.f is stuck unclean for 89072.069127, current state active+remapped+wait_backfill, last acting [26,39,53]
pg 7.32 is stuck unclean for 95058.508838, current state active+remapped+wait_backfill, last acting [48,20,29]
pg 7.50 is stuck unclean for 41018.254911, current state active+remapped+wait_backfill, last acting [37,33,26]
pg 7.1e2 is stuck unclean for 17769.565152, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,21]
pg 7.364 is stuck unclean for 41018.250830, current state active+remapped+wait_backfill, last acting [26,31,14]
pg 7.40 is stuck unclean for 17690.888187, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,49]
pg 10.103 is stuck unclean for 17859.310391, current state down+peering, last acting [30,19,53]
pg 7.94 is stuck unclean for 17859.303440, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,18]
pg 9.153 is stuck unclean for 18047.741550, current state active+remapped+wait_backfill, last acting [32,25,17]
pg 9.382 is stuck unclean for 60158.734087, current state active+undersized+degraded+remapped+wait_backfill, last acting [43,24]
pg 9.66 is stuck unclean since forever, current state down+remapped+peering, last acting [20]
pg 7.136 is stuck unclean for 88230.672888, current state down+remapped+peering, last acting [46]
pg 9.50 is stuck unclean for 41018.203401, current state active+remapped+wait_backfill, last acting [30,38,18]
pg 9.1ee is stuck unclean for 17602.011583, current state active+remapped+wait_backfill, last acting [33,22,30]
pg 9.1f9 is stuck unclean for 64200.225025, current state active+remapped+wait_backfill, last acting [28,11,46]
pg 7.1d0 is stuck unclean for 17769.658211, current state active+undersized+degraded+remapped+backfilling, last acting [28,42]
pg 9.40 is stuck unclean for 17859.281730, current state active+remapped+wait_backfill, last acting [40,21,48]
pg 7.1e5 is stuck unclean for 17602.064037, current state active+remapped+wait_backfill, last acting [40,11,20]
pg 9.1e6 is stuck unclean for 73548.315317, current state active+remapped+wait_backfill, last acting [42,28,21]
pg 7.d is stuck unclean for 17519.396866, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,14]
pg 9.17 is stuck unclean for 25388.590345, current state active+remapped+wait_backfill, last acting [15,51,45]
pg 9.1ab is stuck unclean for 17602.000306, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,11]
pg 9.13d is stuck unclean for 63650.294296, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,11]
pg 9.1ae is stuck unclean for 60072.763752, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,16]
pg 7.154 is stuck unclean for 26785.439771, current state active+remapped+wait_backfill, last acting [35,13,37]
pg 9.10a is stuck unclean for 17841.323281, current state active+undersized+degraded+remapped+wait_backfill, last acting [31,41]
pg 7.237 is stuck unclean for 86489.108177, current state active+remapped+wait_backfill, last acting [23,45,31]
pg 9.ab is stuck unclean for 41018.214309, current state active+remapped+wait_backfill, last acting [30,36,48]
pg 7.128 is stuck unclean for 19531.649034, current state active+remapped+wait_backfill, last acting [53,23,49]
pg 8.3ed is stuck unclean for 17601.972507, current state active+recovery_wait+degraded, last acting [30,15,51]
pg 7.319 is stuck unclean for 88232.331887, current state active+undersized+degraded+remapped+wait_backfill, last acting [51,21]
pg 7.197 is stuck unclean for 17543.165325, current state active+remapped+wait_backfill, last acting [36,40,29]
pg 9.83 is stuck unclean for 59547.664746, current state active+remapped+backfilling, last acting [34,17,16]
pg 7.156 is stuck unclean for 17693.648721, current state active+remapped+wait_backfill, last acting [38,41,54]
pg 7.16f is stuck unclean for 101417.280319, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,11]
pg 8.ae is stuck unclean for 17769.448245, current state down+peering, last acting [39,21,52]
pg 7.af is stuck unclean for 19531.809406, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,24]
pg 9.166 is stuck unclean for 41011.971824, current state active+undersized+degraded+remapped+backfilling, last acting [32,12]
pg 7.b5 is stuck unclean for 24078.339613, current state active+remapped+wait_backfill, last acting [43,46,35]
pg 9.170 is stuck unclean for 73546.766345, current state active+remapped+backfilling, last acting [26,35,45]
pg 7.310 is stuck unclean for 19687.185475, current state active+remapped+wait_backfill, last acting [37,12,30]
pg 9.259 is stuck unclean for 26955.708087, current state active+remapped+wait_backfill, last acting [13,46,15]
pg 9.180 is stuck unclean for 17601.980598, current state active+remapped+wait_backfill, last acting [21,14,48]
pg 9.de is stuck unclean for 73543.374412, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,37]
pg 7.199 is stuck unclean for 41016.008947, current state active+remapped+wait_backfill, last acting [23,27,41]
pg 12.1a5 is stuck unclean for 17190.987255, current state active+recovery_wait+degraded, last acting [40,17,49]
pg 9.3f1 is stuck unclean for 17602.003320, current state active+undersized+degraded+remapped+wait_backfill, last acting [31,52]
pg 7.1a4 is stuck unclean for 17859.305202, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,13]
pg 9.7f is stuck unclean for 73552.511085, current state active+remapped+wait_backfill, last acting [42,30,38]
pg 7.71 is stuck unclean for 89430.995314, current state active+remapped+wait_backfill, last acting [26,34,47]
pg 7.203 is stuck unclean for 17496.694787, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,49]
pg 7.1b3 is stuck unclean for 26228.762719, current state active+remapped+wait_backfill, last acting [33,36,29]
pg 7.21 is stuck unclean for 17693.641700, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,34]
pg 9.3b8 is stuck unclean for 73550.785813, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,42]
pg 7.224 is stuck unclean for 41014.307149, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,40]
pg 9.9c is stuck unclean for 17859.298221, current state active+remapped+wait_backfill, last acting [12,18,42]
pg 9.ff is stuck unclean for 63650.217745, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,43]
pg 9.a2 is stuck unclean for 59507.866010, current state active+remapped+wait_backfill, last acting [40,14,28]
pg 7.a9 is stuck unclean for 85408.171853, current state active+remapped+wait_backfill, last acting [30,41,37]
pg 7.ab is stuck unclean for 88406.021070, current state active+undersized+degraded+remapped+wait_backfill, last acting [21,37]
pg 9.245 is stuck unclean for 41374.573130, current state active+remapped+wait_backfill, last acting [34,19,21]
pg 9.b7 is stuck unclean for 41018.236854, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,14]
pg 7.cc is stuck unclean for 41018.227772, current state active+undersized+degraded+remapped+wait_backfill, last acting [38,11]
pg 7.1a7 is stuck unclean for 17742.655317, current state active+remapped+wait_backfill, last acting [13,31,48]
pg 9.d0 is stuck unclean for 41018.243466, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,19]
pg 7.264 is stuck unclean for 17022.785072, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,33]
pg 7.d2 is stuck unclean for 17755.066843, current state active+remapped+wait_backfill, last acting [12,17,50]
pg 8.277 is stuck unclean for 17858.970795, current state active+remapped+wait_backfill, last acting [21,43,53]
pg 7.f6 is stuck unclean for 17859.203493, current state active+remapped+wait_backfill, last acting [28,43,36]
pg 9.3df is stuck undersized for 9739.449828, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,46]
pg 7.3b8 is stuck undersized for 9765.484068, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,14]
pg 9.3a7 is stuck undersized for 9757.077696, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,30]
pg 9.393 is stuck undersized for 9752.989107, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,45]
pg 9.398 is stuck undersized for 9700.687800, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,12]
pg 9.382 is stuck undersized for 9699.251679, current state active+undersized+degraded+remapped+wait_backfill, last acting [43,24]
pg 7.36d is stuck undersized for 9702.219608, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,29]
pg 7.369 is stuck undersized for 9699.736858, current state active+undersized+degraded+remapped+wait_backfill, last acting [31,15]
pg 9.36d is stuck undersized for 9724.411972, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,21]
pg 9.33d is stuck undersized for 9699.214259, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,20]
pg 7.319 is stuck undersized for 9699.130380, current state active+undersized+degraded+remapped+wait_backfill, last acting [51,21]
pg 7.2d9 is stuck undersized for 9722.703909, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,38]
pg 9.2b2 is stuck undersized for 9737.730587, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,19]
pg 9.2a1 is stuck undersized for 9750.973230, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,49]
pg 7.295 is stuck undersized for 9722.121460, current state active+undersized+degraded+remapped+wait_backfill, last acting [38,21]
pg 7.28b is stuck undersized for 9765.494133, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,14]
pg 9.28c is stuck undersized for 9771.382438, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,15]
pg 7.27a is stuck undersized for 9764.461009, current state active+undersized+degraded+remapped+backfilling, last acting [31,34]
pg 7.264 is stuck undersized for 9773.138936, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,33]
pg 7.255 is stuck undersized for 9702.220755, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,36]
pg 9.24e is stuck undersized for 9740.369078, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,17]
pg 7.235 is stuck undersized for 9724.603148, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,15]
pg 9.3b8 is stuck undersized for 9757.708982, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,42]
pg 7.224 is stuck undersized for 9699.193724, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,40]
pg 7.203 is stuck undersized for 9732.116958, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,49]
pg 9.ff is stuck undersized for 9761.866294, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,43]
pg 9.e3 is stuck undersized for 9770.413753, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,37]
pg 9.de is stuck undersized for 9770.839533, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,37]
pg 9.d0 is stuck undersized for 9773.145391, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,19]
pg 7.cc is stuck undersized for 9700.634069, current state active+undersized+degraded+remapped+wait_backfill, last acting [38,11]
pg 7.b7 is stuck undersized for 9758.444129, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,18]
pg 9.b7 is stuck undersized for 9738.138708, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,14]
pg 7.ab is stuck undersized for 9732.312569, current state active+undersized+degraded+remapped+wait_backfill, last acting [21,37]
pg 7.21 is stuck undersized for 9720.773873, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,34]
pg 9.3f1 is stuck undersized for 9699.731972, current state active+undersized+degraded+remapped+wait_backfill, last acting [31,52]
pg 7.1a4 is stuck undersized for 9699.251894, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,13]
pg 7.d is stuck undersized for 9765.500850, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,14]
pg 9.1ab is stuck undersized for 9735.150544, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,11]
pg 7.af is stuck undersized for 9767.487870, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,24]
pg 9.166 is stuck undersized for 9756.583170, current state active+undersized+degraded+remapped+backfilling, last acting [32,12]
pg 7.16f is stuck undersized for 9762.437507, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,11]
pg 9.10a is stuck undersized for 9767.003353, current state active+undersized+degraded+remapped+wait_backfill, last acting [31,41]
pg 9.1ae is stuck undersized for 9731.158268, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,16]
pg 9.13d is stuck undersized for 9752.686747, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,11]
pg 7.147 is stuck undersized for 9722.454774, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,21]
pg 9.157 is stuck undersized for 9751.888427, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,42]
pg 9.1c1 is stuck undersized for 9772.997466, current state active+undersized+degraded+remapped+backfilling, last acting [33,23]
pg 7.2e is stuck undersized for 9731.011241, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,16]
pg 9.36f is stuck undersized for 9753.963438, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,48]
pg 7.1df is stuck undersized for 9702.529043, current state active+undersized+degraded+remapped+wait_backfill, last acting [38,36]
pg 7.43 is stuck undersized for 9719.407222, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,27]
pg 7.1d0 is stuck undersized for 9766.588187, current state active+undersized+degraded+remapped+backfilling, last acting [28,42]
pg 9.58 is stuck undersized for 9770.782864, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,18]
pg 7.1ec is stuck undersized for 9699.162520, current state active+undersized+degraded+remapped+wait_backfill, last acting [51,30]
pg 7.1ef is stuck undersized for 9699.732777, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,52]
pg 9.46 is stuck undersized for 9699.228129, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,16]
pg 7.40 is stuck undersized for 9765.704693, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,49]
pg 7.1e2 is stuck undersized for 9720.988621, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,21]
pg 7.387 is stuck undersized for 9700.633425, current state active+undersized+degraded+remapped+wait_backfill, last acting [21,29]
pg 7.63 is stuck undersized for 9699.227280, current state active+undersized+degraded+remapped+backfilling, last acting [27,15]
pg 9.67 is stuck undersized for 9773.141351, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,42]
pg 9.8a is stuck undersized for 9722.165070, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,44]
pg 7.94 is stuck undersized for 9699.291230, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,18]
pg 9.3df is stuck degraded for 9739.450326, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,46]
pg 7.3b8 is stuck degraded for 9765.484564, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,14]
pg 9.3a7 is stuck degraded for 9757.078192, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,30]
pg 9.393 is stuck degraded for 9752.989603, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,45]
pg 9.398 is stuck degraded for 9700.688296, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,12]
pg 9.382 is stuck degraded for 9699.252175, current state active+undersized+degraded+remapped+wait_backfill, last acting [43,24]
pg 7.36d is stuck degraded for 9702.220103, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,29]
pg 7.369 is stuck degraded for 9699.737353, current state active+undersized+degraded+remapped+wait_backfill, last acting [31,15]
pg 9.36d is stuck degraded for 9724.412467, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,21]
pg 9.33d is stuck degraded for 9699.214755, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,20]
pg 8.3ed is stuck degraded for 9698.097168, current state active+recovery_wait+degraded, last acting [30,15,51]
pg 7.319 is stuck degraded for 9699.130881, current state active+undersized+degraded+remapped+wait_backfill, last acting [51,21]
pg 8.2d3 is stuck degraded for 9699.709909, current state active+recovery_wait+degraded, last acting [30,33,52]
pg 7.2d9 is stuck degraded for 9722.704415, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,38]
pg 9.2b2 is stuck degraded for 9737.731093, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,19]
pg 9.2a1 is stuck degraded for 9750.973736, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,49]
pg 7.295 is stuck degraded for 9722.121965, current state active+undersized+degraded+remapped+wait_backfill, last acting [38,21]
pg 7.28b is stuck degraded for 9765.494639, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,14]
pg 9.28c is stuck degraded for 9771.382943, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,15]
pg 7.27a is stuck degraded for 9764.461515, current state active+undersized+degraded+remapped+backfilling, last acting [31,34]
pg 7.264 is stuck degraded for 9773.139442, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,33]
pg 7.255 is stuck degraded for 9702.221261, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,36]
pg 9.24e is stuck degraded for 9740.369583, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,17]
pg 7.235 is stuck degraded for 9724.603654, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,15]
pg 9.3b8 is stuck degraded for 9757.709488, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,42]
pg 7.224 is stuck degraded for 9699.194229, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,40]
pg 7.203 is stuck degraded for 9732.117464, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,49]
pg 9.ff is stuck degraded for 9761.866799, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,43]
pg 9.e3 is stuck degraded for 9770.414258, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,37]
pg 9.de is stuck degraded for 9770.840038, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,37]
pg 9.d0 is stuck degraded for 9773.145896, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,19]
pg 7.cc is stuck degraded for 9700.634574, current state active+undersized+degraded+remapped+wait_backfill, last acting [38,11]
pg 8.e is stuck degraded for 9698.250462, current state active+recovery_wait+degraded, last acting [30,43,52]
pg 7.b7 is stuck degraded for 9758.444639, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,18]
pg 9.b7 is stuck degraded for 9738.139218, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,14]
pg 7.ab is stuck degraded for 9732.313079, current state active+undersized+degraded+remapped+wait_backfill, last acting [21,37]
pg 7.21 is stuck degraded for 9720.774383, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,34]
pg 9.3f1 is stuck degraded for 9699.732481, current state active+undersized+degraded+remapped+wait_backfill, last acting [31,52]
pg 7.1a4 is stuck degraded for 9699.252404, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,13]
pg 7.d is stuck degraded for 9765.501359, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,14]
pg 9.1ab is stuck degraded for 9735.151054, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,11]
pg 12.1a5 is stuck degraded for 9752.723460, current state active+recovery_wait+degraded, last acting [40,17,49]
pg 7.af is stuck degraded for 9767.488385, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,24]
pg 9.166 is stuck degraded for 9756.583685, current state active+undersized+degraded+remapped+backfilling, last acting [32,12]
pg 7.16f is stuck degraded for 9762.438022, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,11]
pg 9.10a is stuck degraded for 9767.003867, current state active+undersized+degraded+remapped+wait_backfill, last acting [31,41]
pg 9.1ae is stuck degraded for 9731.158783, current state active+undersized+degraded+remapped+wait_backfill, last acting [42,16]
pg 9.13d is stuck degraded for 9752.687261, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,11]
pg 8.147 is stuck degraded for 9699.706648, current state active+recovery_wait+degraded, last acting [30,52,16]
pg 7.147 is stuck degraded for 9722.455294, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,21]
pg 9.157 is stuck degraded for 9751.888947, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,42]
pg 8.1b8 is stuck degraded for 9685.140845, current state active+recovery_wait+degraded, last acting [40,49,28]
pg 9.1c1 is stuck degraded for 9772.997991, current state active+undersized+degraded+remapped+backfilling, last acting [33,23]
pg 7.2e is stuck degraded for 9731.011766, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,16]
pg 9.36f is stuck degraded for 9753.963962, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,48]
pg 7.1df is stuck degraded for 9702.529567, current state active+undersized+degraded+remapped+wait_backfill, last acting [38,36]
pg 7.43 is stuck degraded for 9719.407746, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,27]
pg 7.1d0 is stuck degraded for 9766.588711, current state active+undersized+degraded+remapped+backfilling, last acting [28,42]
pg 9.58 is stuck degraded for 9770.783388, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,18]
pg 7.1ec is stuck degraded for 9699.163044, current state active+undersized+degraded+remapped+wait_backfill, last acting [51,30]
pg 7.1ef is stuck degraded for 9699.733301, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,52]
pg 9.46 is stuck degraded for 9699.228653, current state active+undersized+degraded+remapped+wait_backfill, last acting [40,16]
pg 7.40 is stuck degraded for 9765.705217, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,49]
pg 7.1e2 is stuck degraded for 9720.989145, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,21]
pg 7.387 is stuck degraded for 9700.633949, current state active+undersized+degraded+remapped+wait_backfill, last acting [21,29]
pg 7.63 is stuck degraded for 9699.227804, current state active+undersized+degraded+remapped+backfilling, last acting [27,15]
pg 9.67 is stuck degraded for 9773.141875, current state active+undersized+degraded+remapped+wait_backfill, last acting [28,42]
pg 9.8a is stuck degraded for 9722.165594, current state active+undersized+degraded+remapped+wait_backfill, last acting [32,44]
pg 7.94 is stuck degraded for 9699.291754, current state active+undersized+degraded+remapped+wait_backfill, last acting [30,18]
pg 7.94 is active+undersized+degraded+remapped+wait_backfill, acting [30,18]
pg 9.8a is active+undersized+degraded+remapped+wait_backfill, acting [32,44]
pg 9.83 is active+remapped+backfilling, acting [34,17,16]
pg 9.7f is active+remapped+wait_backfill, acting [42,30,38]
pg 7.71 is active+remapped+wait_backfill, acting [26,34,47]
pg 11.7a is down+remapped+peering, acting [13,25]
pg 7.79 is active+remapped+wait_backfill, acting [21,22,14]
pg 9.66 is down+remapped+peering, acting [20]
pg 9.67 is active+undersized+degraded+remapped+wait_backfill, acting [28,42]
pg 7.6e is active+remapped+wait_backfill, acting [13,16,43]
pg 7.50 is active+remapped+wait_backfill, acting [37,33,26]
pg 9.1f9 is active+remapped+wait_backfill, acting [28,11,46]
pg 9.50 is active+remapped+wait_backfill, acting [30,38,18]
pg 7.1f5 is active+remapped+wait_backfill, acting [23,27,17]
pg 7.63 is active+undersized+degraded+remapped+backfilling, acting [27,15]
pg 8.6c is down+peering, acting [34,17,48]
pg 7.1e2 is active+undersized+degraded+remapped+wait_backfill, acting [30,21]
pg 9.1ee is active+remapped+wait_backfill, acting [33,22,30]
pg 7.40 is active+undersized+degraded+remapped+wait_backfill, acting [32,49]
pg 9.40 is active+remapped+wait_backfill, acting [40,21,48]
pg 7.1e5 is active+remapped+wait_backfill, acting [40,11,20]
pg 9.1e6 is active+remapped+wait_backfill, acting [42,28,21]
pg 9.46 is active+undersized+degraded+remapped+wait_backfill, acting [40,16]
pg 7.1ef is active+undersized+degraded+remapped+wait_backfill, acting [40,52]
pg 7.1ec is active+undersized+degraded+remapped+wait_backfill, acting [51,30]
pg 9.58 is active+undersized+degraded+remapped+wait_backfill, acting [30,18]
pg 7.1d0 is active+undersized+degraded+remapped+backfilling, acting [28,42]
pg 7.1d1 is active+remapped+wait_backfill, acting [26,52,37]
pg 7.43 is active+undersized+degraded+remapped+wait_backfill, acting [40,27]
pg 9.1d0 is active+remapped+wait_backfill, acting [19,33,17]
pg 8.37 is down+peering, acting [15,40,52]
pg 7.1df is active+undersized+degraded+remapped+wait_backfill, acting [38,36]
pg 9.1cf is active+remapped+wait_backfill, acting [23,24,37]
pg 7.2e is active+undersized+degraded+remapped+wait_backfill, acting [30,16]
pg 7.1cb is active+remapped+wait_backfill, acting [20,47,32]
pg 7.32 is active+remapped+wait_backfill, acting [48,20,29]
pg 9.1c1 is active+undersized+degraded+remapped+backfilling, acting [33,23]
pg 9.38 is down+remapped+peering, acting [46]
pg 7.36 is down+remapped+peering, acting [20]
pg 8.1b8 is active+recovery_wait+degraded, acting [40,49,28]
pg 9.157 is active+undersized+degraded+remapped+wait_backfill, acting [28,42]
pg 9.153 is active+remapped+wait_backfill, acting [32,25,17]
pg 7.147 is active+undersized+degraded+remapped+wait_backfill, acting [28,21]
pg 8.147 is active+recovery_wait+degraded, acting [30,52,16]
pg 8.143 is down+remapped+peering, acting [39,19]
pg 9.142 is active+remapped+backfilling, acting [48,40,46]
pg 9.17 is active+remapped+wait_backfill, acting [15,51,45]
pg 9.13d is active+undersized+degraded+remapped+wait_backfill, acting [40,11]
pg 7.136 is down+remapped+peering, acting [46]
pg 9.1ae is active+undersized+degraded+remapped+wait_backfill, acting [42,16]
pg 7.1b3 is active+remapped+wait_backfill, acting [33,36,29]
pg 9.10a is active+undersized+degraded+remapped+wait_backfill, acting [31,41]
pg 9.1af is active+remapped+wait_backfill, acting [17,45,21]
pg 7.128 is active+remapped+wait_backfill, acting [53,23,49]
pg 10.103 is down+peering, acting [30,19,53]
pg 7.111 is active+remapped+wait_backfill, acting [23,45,29]
pg 7.154 is active+remapped+wait_backfill, acting [35,13,37]
pg 7.156 is active+remapped+wait_backfill, acting [38,41,54]
pg 7.16f is active+undersized+degraded+remapped+wait_backfill, acting [32,11]
pg 9.166 is active+undersized+degraded+remapped+backfilling, acting [32,12]
pg 9.16c is active+remapped+backfilling, acting [14,52,51]
pg 9.170 is active+remapped+backfilling, acting [26,35,45]
pg 7.1a7 is active+remapped+wait_backfill, acting [13,31,48]
pg 9.180 is active+remapped+wait_backfill, acting [21,14,48]
pg 7.f is active+remapped+wait_backfill, acting [26,39,53]
pg 7.199 is active+remapped+wait_backfill, acting [23,27,41]
pg 12.1a5 is active+recovery_wait+degraded, acting [40,17,49]
pg 7.197 is active+remapped+wait_backfill, acting [36,40,29]
pg 9.1ab is active+undersized+degraded+remapped+wait_backfill, acting [28,11]
pg 7.d is active+undersized+degraded+remapped+wait_backfill, acting [42,14]
pg 7.1a4 is active+undersized+degraded+remapped+wait_backfill, acting [30,13]
pg 7.21 is active+undersized+degraded+remapped+wait_backfill, acting [28,34]
pg 9.9c is active+remapped+wait_backfill, acting [12,18,42]
pg 9.a2 is active+remapped+wait_backfill, acting [40,14,28]
pg 7.af is active+undersized+degraded+remapped+wait_backfill, acting [40,24]
pg 7.a9 is active+remapped+wait_backfill, acting [30,41,37]
pg 7.ab is active+undersized+degraded+remapped+wait_backfill, acting [21,37]
pg 9.ab is active+remapped+wait_backfill, acting [30,36,48]
pg 8.ae is down+peering, acting [39,21,52]
pg 9.b7 is active+undersized+degraded+remapped+wait_backfill, acting [32,14]
pg 7.b5 is active+remapped+wait_backfill, acting [43,46,35]
pg 9.b9 is active+remapped+wait_backfill, acting [13,18,43]
pg 7.b7 is active+undersized+degraded+remapped+wait_backfill, acting [42,18]
pg 8.e is active+recovery_wait+degraded, acting [30,43,52]
pg 7.cc is active+undersized+degraded+remapped+wait_backfill, acting [38,11]
pg 9.c1 is active+remapped+wait_backfill, acting [31,34,30]
pg 9.d0 is active+undersized+degraded+remapped+wait_backfill, acting [28,19]
pg 9.de is active+undersized+degraded+remapped+wait_backfill, acting [30,37]
pg 7.d2 is active+remapped+wait_backfill, acting [12,17,50]
pg 9.e3 is active+undersized+degraded+remapped+wait_backfill, acting [30,37]
pg 9.e2 is active+remapped+backfilling, acting [12,32,41]
pg 7.ec is active+remapped+wait_backfill, acting [38,16,42]
pg 7.f6 is active+remapped+wait_backfill, acting [28,43,36]
pg 9.ff is active+undersized+degraded+remapped+wait_backfill, acting [30,43]
pg 9.20e is active+remapped+wait_backfill, acting [12,36,51]
pg 7.203 is active+undersized+degraded+remapped+wait_backfill, acting [28,49]
pg 7.20e is active+remapped+wait_backfill, acting [31,54,41]
pg 7.210 is incomplete, acting [22,49,15] (reducing pool seafile-fs min_size from 2 may help; search ceph.com/docs for 'incomplete')
pg 9.218 is active+remapped+wait_backfill, acting [35,29,41]
pg 7.217 is active+remapped+wait_backfill, acting [13,21,47]
pg 9.22c is active+remapped+wait_backfill, acting [17,52,23]
pg 7.224 is active+undersized+degraded+remapped+wait_backfill, acting [32,40]
pg 7.227 is active+remapped+wait_backfill, acting [40,52,45]
pg 9.23e is down+remapped+peering, acting [18]
pg 9.23b is active+remapped+wait_backfill, acting [12,18,38]
pg 7.235 is active+undersized+degraded+remapped+wait_backfill, acting [32,15]
pg 7.237 is active+remapped+wait_backfill, acting [23,45,31]
pg 9.24e is active+undersized+degraded+remapped+wait_backfill, acting [42,17]
pg 9.24b is active+remapped+wait_backfill, acting [16,21,32]
pg 9.245 is active+remapped+wait_backfill, acting [34,19,21]
pg 9.243 is incomplete, acting [34,47,26] (reducing pool seafile-blocks min_size from 2 may help; search ceph.com/docs for 'incomplete')
pg 7.255 is active+undersized+degraded+remapped+wait_backfill, acting [32,36]
pg 9.259 is active+remapped+wait_backfill, acting [13,46,15]
pg 7.25d is active+remapped+wait_backfill, acting [32,20,31]
pg 7.264 is active+undersized+degraded+remapped+wait_backfill, acting [28,33]
pg 7.272 is active+remapped+wait_backfill, acting [20,15,36]
pg 8.277 is active+remapped+wait_backfill, acting [21,43,53]
pg 7.27a is active+undersized+degraded+remapped+backfilling, acting [31,34]
pg 9.28c is active+undersized+degraded+remapped+wait_backfill, acting [40,15]
pg 7.28b is active+undersized+degraded+remapped+wait_backfill, acting [40,14]
pg 7.295 is active+undersized+degraded+remapped+wait_backfill, acting [38,21]
pg 9.2a8 is active+remapped+wait_backfill, acting [15,17,44]
pg 9.2a1 is active+undersized+degraded+remapped+wait_backfill, acting [28,49]
pg 9.2b8 is active+remapped+backfilling, acting [18,21,50]
pg 9.2b7 is active+remapped+wait_backfill, acting [23,51,38]
pg 7.2b9 is active+remapped+wait_backfill, acting [32,47,50]
pg 9.2b5 is active+remapped+wait_backfill, acting [32,29,33]
pg 9.2b2 is active+undersized+degraded+remapped+wait_backfill, acting [40,19]
pg 9.2b0 is active+remapped+wait_backfill, acting [52,11,21]
pg 7.2d6 is active+remapped+wait_backfill, acting [16,15,36]
pg 7.2d9 is active+undersized+degraded+remapped+wait_backfill, acting [32,38]
pg 8.2d3 is active+recovery_wait+degraded, acting [30,33,52]
pg 7.30c is active+remapped+wait_backfill, acting [12,39,16]
pg 7.310 is active+remapped+wait_backfill, acting [37,12,30]
pg 7.319 is active+undersized+degraded+remapped+wait_backfill, acting [51,21]
pg 9.33d is active+undersized+degraded+remapped+wait_backfill, acting [42,20]
pg 9.34c is active+remapped+wait_backfill, acting [21,18,40]
pg 7.34d is active+remapped+wait_backfill, acting [48,14,22]
pg 9.352 is active+remapped+wait_backfill, acting [36,54,30]
pg 9.36f is active+undersized+degraded+remapped+wait_backfill, acting [30,48]
pg 9.36d is active+undersized+degraded+remapped+wait_backfill, acting [32,21]
pg 7.364 is active+remapped+wait_backfill, acting [26,31,14]
pg 9.368 is active+remapped+wait_backfill, acting [21,25,19]
pg 7.369 is active+undersized+degraded+remapped+wait_backfill, acting [31,15]
pg 9.363 is active+remapped+wait_backfill, acting [40,16,32]
pg 7.36d is active+undersized+degraded+remapped+wait_backfill, acting [32,29]
pg 7.387 is active+undersized+degraded+remapped+wait_backfill, acting [21,29]
pg 7.38b is active+remapped+wait_backfill, acting [13,34,42]
pg 9.382 is active+undersized+degraded+remapped+wait_backfill, acting [43,24]
pg 9.398 is active+undersized+degraded+remapped+wait_backfill, acting [32,12]
pg 7.399 is down+remapped+peering, acting [21]
pg 9.393 is active+undersized+degraded+remapped+wait_backfill, acting [30,45]
pg 9.3ac is down+remapped+peering, acting [36]
pg 7.3a5 is active+remapped+wait_backfill, acting [36,28,33]
pg 7.3a7 is active+remapped+wait_backfill, acting [38,24,37]
pg 9.3a7 is active+undersized+degraded+remapped+wait_backfill, acting [42,30]
pg 9.3b8 is active+undersized+degraded+remapped+wait_backfill, acting [30,42]
pg 7.3b8 is active+undersized+degraded+remapped+wait_backfill, acting [42,14]
pg 9.3b1 is active+remapped+wait_backfill, acting [35,40,14]
pg 7.3c8 is active+remapped+wait_backfill, acting [30,35,40]
pg 9.3df is active+undersized+degraded+remapped+wait_backfill, acting [28,46]
pg 9.3db is active+remapped+wait_backfill, acting [40,33,52]
pg 8.3ed is active+recovery_wait+degraded, acting [30,15,51]
pg 8.3ec is down+peering, acting [22,40,49]
pg 9.3e8 is active+remapped+wait_backfill, acting [28,53,27]
pg 9.3ff is active+remapped+wait_backfill, acting [48,53,33]
pg 9.3f1 is active+undersized+degraded+remapped+wait_backfill, acting [31,52]
1 ops are blocked > 8388.61 sec on osd.22
1 ops are blocked > 16777.2 sec on osd.18
6 ops are blocked > 16777.2 sec on osd.15
3 ops are blocked > 8388.61 sec on osd.15
2 ops are blocked > 4194.3 sec on osd.15
1 ops are blocked > 2097.15 sec on osd.15
1 ops are blocked > 16777.2 sec on osd.34
1 ops are blocked > 16777.2 sec on osd.36
8 ops are blocked > 16777.2 sec on osd.46
3 ops are blocked > 8388.61 sec on osd.46
2 ops are blocked > 4194.3 sec on osd.46
6 osds have slow requests
recovery 667605/71152293 objects degraded (0.938%)
recovery 1564114/71152293 objects misplaced (2.198%)
too many PGs per OSD (314 > max 300)

Attachment: smime.p7s
Description: S/MIME cryptographic signature

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux