Hi Joseph
thanks for helping with this. Here are the outputs:
ceph osd df tree:
ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP
META AVAIL %USE VAR PGS STATUS TYPE NAME
-6 4.32092 - 3.9 TiB 208 GiB 147 GiB 43 GiB
17 GiB 3.7 TiB 5.16 0.07 - root ssd
-3 0.48010 - 447 GiB 23 GiB 16 GiB 5.2 GiB
1.5 GiB 424 GiB 5.15 0.07 - host storage4-osd1-ssd
350 ssd 0.48010 1.00000 447 GiB 23 GiB 16 GiB 5.2 GiB
1.5 GiB 424 GiB 5.15 0.07 43 up osd.350
-76 0.48010 - 447 GiB 23 GiB 16 GiB 4.8 GiB
1.9 GiB 424 GiB 5.16 0.07 - host storage4-osd10-ssd
579 ssd 0.48010 1.00000 447 GiB 23 GiB 16 GiB 4.8 GiB
1.9 GiB 424 GiB 5.16 0.07 43 up osd.579
-82 0.48010 - 447 GiB 23 GiB 16 GiB 5.4 GiB
1.3 GiB 424 GiB 5.17 0.07 - host storage4-osd11-ssd
585 ssd 0.48010 1.00000 447 GiB 23 GiB 16 GiB 5.4 GiB
1.3 GiB 424 GiB 5.17 0.07 43 up osd.585
-85 0.48010 - 447 GiB 23 GiB 16 GiB 5.2 GiB
1.5 GiB 424 GiB 5.14 0.07 - host storage4-osd12-ssd
586 ssd 0.48010 1.00000 447 GiB 23 GiB 16 GiB 5.2 GiB
1.5 GiB 424 GiB 5.14 0.07 43 up osd.586
-7 0.48010 - 447 GiB 23 GiB 16 GiB 2.7 GiB
3.9 GiB 424 GiB 5.16 0.07 - host storage4-osd2-ssd
355 ssd 0.48010 1.00000 447 GiB 23 GiB 16 GiB 2.7 GiB
3.9 GiB 424 GiB 5.16 0.07 44 up osd.355
-11 0.48010 - 447 GiB 24 GiB 16 GiB 6.0 GiB
1.2 GiB 423 GiB 5.29 0.07 - host storage4-osd3-ssd
315 ssd 0.48010 1.00000 447 GiB 24 GiB 16 GiB 6.0 GiB
1.2 GiB 423 GiB 5.29 0.07 42 up osd.315
-73 0.48010 - 447 GiB 23 GiB 16 GiB 4.4 GiB
2.1 GiB 424 GiB 5.12 0.07 - host storage4-osd6-ssd
566 ssd 0.48010 1.00000 447 GiB 23 GiB 16 GiB 4.4 GiB
2.1 GiB 424 GiB 5.12 0.07 43 up osd.566
-70 0.48010 - 447 GiB 23 GiB 16 GiB 5.0 GiB
1.6 GiB 424 GiB 5.14 0.07 - host storage4-osd7-ssd
565 ssd 0.48010 1.00000 447 GiB 23 GiB 16 GiB 5.0 GiB
1.6 GiB 424 GiB 5.14 0.07 43 up osd.565
-79 0.48010 - 447 GiB 23 GiB 16 GiB 4.7 GiB
1.9 GiB 424 GiB 5.13 0.07 - host storage4-osd8-ssd
584 ssd 0.48010 1.00000 447 GiB 23 GiB 16 GiB 4.7 GiB
1.9 GiB 424 GiB 5.13 0.07 43 up osd.584
-1 9056.38672 - 8.0 PiB 6.1 PiB 6.1 PiB 875 MiB
17 TiB 1.9 PiB 76.02 1.00 - root default
-64 566.02417 - 515 TiB 391 TiB 390 TiB 58 MiB
1.0 TiB 123 TiB 76.05 1.00 - host storage4-osd1
0 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.1 MiB
36 GiB 3.3 TiB 79.86 1.05 196 up osd.0
65 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.7 MiB
17 GiB 1.5 TiB 79.93 1.05 87 up osd.65
229 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.5 MiB
16 GiB 1.5 TiB 79.91 1.05 87 up osd.229
230 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 254 KiB
16 GiB 1.5 TiB 79.87 1.05 87 up osd.230
231 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.4 MiB
16 GiB 1.5 TiB 79.88 1.05 87 up osd.231
232 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 914 KiB
16 GiB 1.5 TiB 79.89 1.05 87 up osd.232
233 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.8 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.233
234 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.3 MiB
16 GiB 1.5 TiB 79.96 1.05 87 up osd.234
235 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 3.2 MiB
16 GiB 1.5 TiB 79.05 1.04 86 up osd.235
236 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.2 MiB
16 GiB 1.5 TiB 79.91 1.05 87 up osd.236
237 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.3 MiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.237
238 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.6 MiB
16 GiB 1.5 TiB 79.95 1.05 87 up osd.238
265 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.81 0.96 195 up osd.265
275 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.275
294 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.294
311 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.311
329 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.329
345 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.345
360 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.77 0.96 195 up osd.360
384 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.76 0.96 195 up osd.384
400 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.400
416 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.416
426 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.426
428 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.5 MiB
37 GiB 3.3 TiB 79.90 1.05 196 up osd.428
429 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.4 MiB
37 GiB 3.3 TiB 79.85 1.05 196 up osd.429
456 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.456
488 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.19 0.96 196 up osd.488
509 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 2.2 MiB
28 GiB 2.6 TiB 79.68 1.05 152 up osd.509
525 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 3.6 MiB
33 GiB 3.0 TiB 79.32 1.04 173 up osd.525
545 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 3.7 MiB
33 GiB 2.9 TiB 79.78 1.05 174 up osd.545
559 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.5 MiB
36 GiB 3.3 TiB 79.87 1.05 196 up osd.559
568 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.2 MiB
36 GiB 3.4 TiB 79.45 1.05 195 up osd.568
601 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 469 KiB
36 GiB 3.4 TiB 79.44 1.05 195 up osd.601
605 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.4 MiB
37 GiB 3.3 TiB 79.87 1.05 196 up osd.605
632 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.80 0.96 195 up osd.632
648 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.648
664 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.664
680 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.680
-40 566.02417 - 515 TiB 392 TiB 390 TiB 58 MiB
1.0 TiB 123 TiB 76.05 1.00 - host storage4-osd10
15 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.6 MiB
17 GiB 1.5 TiB 79.94 1.05 87 up osd.15
16 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.6 MiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.16
37 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.9 MiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.37
38 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.2 MiB
17 GiB 1.5 TiB 79.93 1.05 87 up osd.38
70 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 140 KiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.70
85 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.8 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.85
99 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 831 KiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.99
114 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.6 MiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.114
131 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 992 KiB
17 GiB 1.5 TiB 79.94 1.05 87 up osd.131
144 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 3.3 MiB
16 GiB 1.5 TiB 79.04 1.04 86 up osd.144
152 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 2.6 MiB
16 GiB 1.5 TiB 79.02 1.04 86 up osd.152
257 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.80 0.96 195 up osd.257
272 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.4 TiB 73.17 0.96 196 up osd.272
292 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.292
303 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.303
320 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.320
336 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.336
353 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.353
371 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.80 0.96 195 up osd.371
386 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.386
402 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 33
GiB 4.5 TiB 72.80 0.96 195 up osd.402
418 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.418
462 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 689 KiB
36 GiB 3.3 TiB 79.85 1.05 196 up osd.462
466 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.8 MiB
37 GiB 3.4 TiB 79.42 1.05 195 up osd.466
472 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.472
487 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.82 0.96 195 up osd.487
499 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.499
511 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 5.3 MiB
28 GiB 2.6 TiB 79.67 1.05 152 up osd.511
528 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 4.7 MiB
33 GiB 2.9 TiB 79.83 1.05 174 up osd.528
548 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 261 KiB
32 GiB 2.9 TiB 79.77 1.05 174 up osd.548
556 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.7 MiB
36 GiB 3.3 TiB 79.87 1.05 196 up osd.556
576 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.1 MiB
36 GiB 3.3 TiB 79.84 1.05 196 up osd.576
591 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 519 KiB
37 GiB 3.4 TiB 79.45 1.05 195 up osd.591
615 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.3 MiB
37 GiB 3.3 TiB 79.84 1.05 196 up osd.615
618 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.0 MiB
36 GiB 3.4 TiB 79.47 1.05 195 up osd.618
638 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.638
653 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.653
669 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.81 0.96 195 up osd.669
-43 566.02417 - 515 TiB 391 TiB 390 TiB 57 MiB
1.0 TiB 123 TiB 76.03 1.00 - host storage4-osd11
17 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.2 MiB
16 GiB 1.5 TiB 79.95 1.05 87 up osd.17
18 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.7 MiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.18
39 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.1 MiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.39
40 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 3.9 MiB
16 GiB 1.5 TiB 78.98 1.04 86 up osd.40
67 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.8 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.67
80 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 3.8 MiB
16 GiB 1.5 TiB 79.00 1.04 86 up osd.80
108 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.0 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.108
121 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1010 KiB
16 GiB 1.5 TiB 79.95 1.05 87 up osd.121
137 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.8 MiB
16 GiB 1.5 TiB 79.88 1.05 87 up osd.137
149 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.4 MiB
16 GiB 1.5 TiB 79.91 1.05 87 up osd.149
211 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 3.5 MiB
16 GiB 1.5 TiB 79.02 1.04 86 up osd.211
253 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.4 TiB 73.17 0.96 196 up osd.253
258 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.258
281 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.281
309 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.19 0.96 196 up osd.309
328 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.19 0.96 196 up osd.328
344 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.344
366 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.366
370 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.14 0.96 196 up osd.370
387 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.387
407 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.15 0.96 196 up osd.407
424 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.424
433 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 33
GiB 4.5 TiB 72.80 0.96 195 up osd.433
461 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.2 MiB
36 GiB 3.3 TiB 79.84 1.05 196 up osd.461
465 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 15 KiB
36 GiB 3.3 TiB 79.82 1.05 196 up osd.465
478 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.4 TiB 73.18 0.96 196 up osd.478
497 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 72.83 0.96 195 up osd.497
510 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 1.1 MiB
29 GiB 2.6 TiB 79.65 1.05 152 up osd.510
529 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 2.5 MiB
32 GiB 3.0 TiB 79.34 1.04 173 up osd.529
543 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 5.5 MiB
32 GiB 3.0 TiB 79.31 1.04 173 up osd.543
558 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.5 MiB
36 GiB 3.4 TiB 79.49 1.05 195 up osd.558
577 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.4 MiB
36 GiB 3.4 TiB 79.43 1.05 195 up osd.577
592 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.4 MiB
36 GiB 3.4 TiB 79.44 1.05 195 up osd.592
614 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.8 MiB
36 GiB 3.4 TiB 79.43 1.05 195 up osd.614
619 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.6 MiB
37 GiB 3.3 TiB 79.85 1.05 196 up osd.619
637 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 72.83 0.96 195 up osd.637
652 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.652
668 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.668
-61 566.02417 - 515 TiB 392 TiB 391 TiB 60 MiB
1.0 TiB 123 TiB 76.07 1.00 - host storage4-osd12
158 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.5 MiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.158
159 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.8 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.159
161 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.8 MiB
16 GiB 1.5 TiB 79.95 1.05 87 up osd.161
163 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.4 MiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.163
166 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.9 MiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.166
168 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.2 MiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.168
173 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.5 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.173
178 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 873 KiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.178
183 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.6 MiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.183
188 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.3 MiB
16 GiB 1.5 TiB 79.91 1.05 87 up osd.188
192 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.0 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.192
263 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.4 TiB 73.16 0.96 196 up osd.263
266 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.266
288 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.288
302 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.80 0.96 195 up osd.302
321 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.14 0.96 196 up osd.321
337 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.4 TiB 73.16 0.96 196 up osd.337
354 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.19 0.96 196 up osd.354
374 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 33
GiB 4.4 TiB 73.19 0.96 196 up osd.374
392 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.392
405 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.405
421 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.421
442 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.442
459 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.3 MiB
36 GiB 3.4 TiB 79.43 1.05 195 up osd.459
460 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.8 MiB
37 GiB 3.4 TiB 79.43 1.05 195 up osd.460
483 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.483
513 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 3.3 MiB
28 GiB 2.6 TiB 79.70 1.05 152 up osd.513
526 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 3.4 MiB
32 GiB 2.9 TiB 79.78 1.05 174 up osd.526
540 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 2.6 MiB
32 GiB 3.0 TiB 79.32 1.04 173 up osd.540
561 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.8 MiB
36 GiB 3.4 TiB 79.46 1.05 195 up osd.561
575 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.5 MiB
36 GiB 3.3 TiB 79.83 1.05 196 up osd.575
593 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 893 KiB
38 GiB 3.3 TiB 79.87 1.05 196 up osd.593
616 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.9 MiB
36 GiB 3.4 TiB 79.43 1.05 195 up osd.616
620 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.1 MiB
36 GiB 3.4 TiB 79.44 1.05 195 up osd.620
630 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.78 0.96 195 up osd.630
647 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.647
662 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.662
677 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.81 0.96 195 up osd.677
-55 566.02417 - 515 TiB 391 TiB 390 TiB 52 MiB
1.0 TiB 123 TiB 76.02 1.00 - host storage4-osd13
33 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 3.6 MiB
16 GiB 1.5 TiB 78.99 1.04 86 up osd.33
34 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.8 MiB
16 GiB 1.5 TiB 79.95 1.05 87 up osd.34
35 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.8 MiB
16 GiB 1.5 TiB 79.95 1.05 87 up osd.35
36 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 1.4 MiB
16 GiB 1.5 TiB 79.05 1.04 86 up osd.36
63 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 79 KiB
16 GiB 1.5 TiB 79.98 1.05 87 up osd.63
86 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 3.8 MiB
16 GiB 1.5 TiB 79.05 1.04 86 up osd.86
100 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.4 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.100
119 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.4 MiB
16 GiB 1.5 TiB 79.95 1.05 87 up osd.119
142 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.0 MiB
16 GiB 1.5 TiB 79.95 1.05 87 up osd.142
153 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.9 MiB
16 GiB 1.5 TiB 79.96 1.05 87 up osd.153
154 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 3.2 MiB
16 GiB 1.5 TiB 79.04 1.04 86 up osd.154
255 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.255
279 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.279
300 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.78 0.96 195 up osd.300
305 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.80 0.96 195 up osd.305
324 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.14 0.96 196 up osd.324
338 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.338
356 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 33
GiB 4.4 TiB 73.17 0.96 196 up osd.356
368 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.15 0.96 196 up osd.368
382 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.81 0.96 195 up osd.382
397 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.397
412 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.412
457 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.3 MiB
36 GiB 3.4 TiB 79.42 1.05 195 up osd.457
458 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.9 MiB
37 GiB 3.3 TiB 79.81 1.05 196 up osd.458
475 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.4 TiB 73.18 0.96 196 up osd.475
491 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.491
501 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.4 TiB 73.19 0.96 196 up osd.501
504 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 1.3 MiB
28 GiB 2.6 TiB 79.69 1.05 152 up osd.504
530 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 223 KiB
32 GiB 2.9 TiB 79.79 1.05 174 up osd.530
536 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 2.2 MiB
33 GiB 2.9 TiB 79.80 1.05 174 up osd.536
554 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 932 KiB
36 GiB 3.4 TiB 79.48 1.05 195 up osd.554
581 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.6 MiB
36 GiB 3.3 TiB 79.86 1.05 196 up osd.581
594 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.4 MiB
36 GiB 3.4 TiB 79.46 1.05 195 up osd.594
622 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 764 KiB
36 GiB 3.3 TiB 79.82 1.05 196 up osd.622
626 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.0 MiB
37 GiB 3.4 TiB 79.48 1.05 195 up osd.626
644 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.644
661 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.661
679 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.679
-46 566.02417 - 515 TiB 391 TiB 390 TiB 52 MiB
1.0 TiB 124 TiB 75.87 1.00 - host storage4-osd14
19 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.3 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.19
20 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.7 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.20
41 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.7 MiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.41
42 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.5 MiB
16 GiB 1.5 TiB 79.91 1.05 87 up osd.42
72 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.8 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.72
75 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.0 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.75
89 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.5 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.89
102 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.2 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.102
115 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.5 MiB
16 GiB 1.5 TiB 79.89 1.05 87 up osd.115
128 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 244 KiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.128
140 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.3 MiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.140
247 hdd 14.00052 1.00000 13 TiB 9.3 TiB 9.3 TiB 0 B 25
GiB 3.4 TiB 72.98 0.96 152 up osd.247
268 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.268
285 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.285
307 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.80 0.96 195 up osd.307
308 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.308
326 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.326
343 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.343
365 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.365
373 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.373
388 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.19 0.96 196 up osd.388
403 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.78 0.96 195 up osd.403
419 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.19 0.96 196 up osd.419
437 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.437
451 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.5 MiB
36 GiB 3.3 TiB 79.86 1.05 196 up osd.451
454 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.8 MiB
36 GiB 3.3 TiB 79.83 1.05 196 up osd.454
481 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.481
521 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 5.2 MiB
32 GiB 2.9 TiB 79.78 1.05 174 up osd.521
532 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.77 0.96 195 up osd.532
547 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 4.4 MiB
32 GiB 2.9 TiB 79.77 1.05 174 up osd.547
560 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.4 MiB
36 GiB 3.4 TiB 79.45 1.05 195 up osd.560
580 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.9 MiB
36 GiB 3.4 TiB 79.46 1.05 195 up osd.580
595 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 215 KiB
37 GiB 3.4 TiB 79.46 1.05 195 up osd.595
624 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.0 MiB
36 GiB 3.4 TiB 79.43 1.05 195 up osd.624
628 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 651 KiB
37 GiB 3.4 TiB 79.43 1.05 195 up osd.628
643 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.643
660 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.660
676 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.80 0.96 195 up osd.676
-52 566.02417 - 515 TiB 391 TiB 390 TiB 48 MiB
1.0 TiB 123 TiB 76.04 1.00 - host storage4-osd15
29 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.7 MiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.29
30 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.8 MiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.30
31 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 59 KiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.31
32 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.9 MiB
17 GiB 1.5 TiB 79.92 1.05 87 up osd.32
69 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.5 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.69
83 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.9 MiB
17 GiB 1.5 TiB 79.93 1.05 87 up osd.83
97 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 349 KiB
16 GiB 1.5 TiB 79.91 1.05 87 up osd.97
112 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.7 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.112
124 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.124
139 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.8 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.139
150 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.4 MiB
16 GiB 1.5 TiB 79.96 1.05 87 up osd.150
167 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 3.5 MiB
16 GiB 1.5 TiB 78.98 1.04 86 up osd.167
254 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.254
271 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 72.84 0.96 195 up osd.271
283 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.283
314 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.78 0.96 195 up osd.314
331 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 72.83 0.96 195 up osd.331
347 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.347
363 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.15 0.96 196 up osd.363
378 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.378
394 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.394
409 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.409
431 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.81 0.96 195 up osd.431
439 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.8 MiB
37 GiB 3.3 TiB 79.82 1.05 196 up osd.439
448 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.4 MiB
37 GiB 3.4 TiB 79.45 1.05 195 up osd.448
477 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.19 0.96 196 up osd.477
494 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.494
514 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 4.4 MiB
29 GiB 2.6 TiB 79.70 1.05 152 up osd.514
527 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 496 KiB
33 GiB 2.9 TiB 79.78 1.05 174 up osd.527
542 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 170 KiB
33 GiB 2.9 TiB 79.79 1.05 174 up osd.542
549 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 576 KiB
37 GiB 3.4 TiB 79.47 1.05 195 up osd.549
582 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.7 MiB
36 GiB 3.4 TiB 79.45 1.05 195 up osd.582
596 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.8 MiB
36 GiB 3.3 TiB 79.86 1.05 196 up osd.596
623 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.6 MiB
36 GiB 3.3 TiB 79.85 1.05 196 up osd.623
627 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.1 MiB
36 GiB 3.4 TiB 79.44 1.05 195 up osd.627
634 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.78 0.96 195 up osd.634
650 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.650
666 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.666
-49 566.02417 - 515 TiB 392 TiB 391 TiB 54 MiB
1.0 TiB 123 TiB 76.08 1.00 - host storage4-osd16
25 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.2 MiB
16 GiB 1.5 TiB 79.96 1.05 87 up osd.25
26 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.8 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.26
27 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.8 MiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.27
28 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.6 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.28
71 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 592 KiB
16 GiB 1.5 TiB 79.91 1.05 87 up osd.71
84 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.4 MiB
16 GiB 1.5 TiB 79.96 1.05 87 up osd.84
96 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.4 MiB
16 GiB 1.5 TiB 79.91 1.05 87 up osd.96
110 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 572 KiB
16 GiB 1.5 TiB 79.87 1.05 87 up osd.110
125 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.6 MiB
16 GiB 1.5 TiB 79.89 1.05 87 up osd.125
134 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.7 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.134
146 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.6 MiB
16 GiB 1.5 TiB 79.97 1.05 87 up osd.146
244 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.81 0.96 195 up osd.244
267 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.267
280 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.4 TiB 73.17 0.96 196 up osd.280
296 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.82 0.96 195 up osd.296
312 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.19 0.96 196 up osd.312
330 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.330
346 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.15 0.96 196 up osd.346
357 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.357
377 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.81 0.96 195 up osd.377
393 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.393
410 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.410
427 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.21 0.96 196 up osd.427
434 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.9 MiB
35 GiB 3.4 TiB 79.42 1.05 195 up osd.434
436 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 283 KiB
37 GiB 3.3 TiB 79.83 1.05 196 up osd.436
471 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.8 MiB
35 GiB 3.4 TiB 79.45 1.05 195 up osd.471
476 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.476
493 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.493
517 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 106 KiB
28 GiB 2.6 TiB 79.66 1.05 152 up osd.517
523 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 5.6 MiB
31 GiB 2.9 TiB 79.76 1.05 174 up osd.523
534 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 5.3 MiB
31 GiB 2.9 TiB 79.77 1.05 174 up osd.534
550 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 359 KiB
35 GiB 3.4 TiB 79.43 1.05 195 up osd.550
597 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.7 MiB
35 GiB 3.4 TiB 79.45 1.05 195 up osd.597
625 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.4 MiB
37 GiB 3.3 TiB 79.86 1.05 196 up osd.625
629 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 738 KiB
35 GiB 3.4 TiB 79.41 1.05 195 up osd.629
633 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.633
649 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.78 0.96 195 up osd.649
665 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.81 0.96 195 up osd.665
-19 566.02417 - 515 TiB 391 TiB 390 TiB 61 MiB
1.0 TiB 124 TiB 76.01 1.00 - host storage4-osd2
1 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.0 MiB
36 GiB 3.4 TiB 79.46 1.05 195 up osd.1
3 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.7 MiB
16 GiB 1.5 TiB 79.91 1.05 87 up osd.3
4 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 3.7 MiB
16 GiB 1.5 TiB 79.00 1.04 86 up osd.4
43 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.2 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.43
44 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 3.7 MiB
16 GiB 1.5 TiB 79.00 1.04 86 up osd.44
61 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 3.7 MiB
16 GiB 1.5 TiB 79.04 1.04 86 up osd.61
76 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.4 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.76
88 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.3 MiB
16 GiB 1.5 TiB 79.89 1.05 87 up osd.88
103 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.8 MiB
16 GiB 1.5 TiB 79.97 1.05 87 up osd.103
116 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.5 MiB
16 GiB 1.5 TiB 79.97 1.05 87 up osd.116
129 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.7 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.129
141 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 370 KiB
16 GiB 1.5 TiB 79.03 1.04 86 up osd.141
249 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.249
270 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.19 0.96 196 up osd.270
289 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.289
291 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.291
298 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.298
313 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.313
332 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.332
348 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.348
380 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.380
396 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.78 0.96 195 up osd.396
413 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.78 0.96 195 up osd.413
430 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.0 MiB
36 GiB 3.4 TiB 79.44 1.05 195 up osd.430
432 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.3 MiB
37 GiB 3.3 TiB 79.83 1.05 196 up osd.432
438 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.438
479 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.20 0.96 196 up osd.479
496 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.496
505 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 3.3 MiB
28 GiB 2.6 TiB 79.66 1.05 152 up osd.505
524 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 750 KiB
32 GiB 2.9 TiB 79.80 1.05 174 up osd.524
538 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 5.0 MiB
32 GiB 2.9 TiB 79.80 1.05 174 up osd.538
557 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.4 MiB
37 GiB 3.4 TiB 79.45 1.05 195 up osd.557
569 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.2 MiB
36 GiB 3.3 TiB 79.85 1.05 196 up osd.569
598 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.6 MiB
36 GiB 3.4 TiB 79.43 1.05 195 up osd.598
604 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.5 MiB
36 GiB 3.4 TiB 79.43 1.05 195 up osd.604
636 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.78 0.96 195 up osd.636
655 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.80 0.96 195 up osd.655
670 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.670
-22 566.02417 - 515 TiB 391 TiB 390 TiB 49 MiB
1.0 TiB 123 TiB 76.04 1.00 - host storage4-osd3
2 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.3 MiB
36 GiB 3.3 TiB 79.85 1.05 196 up osd.2
21 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 376 KiB
17 GiB 1.5 TiB 79.97 1.05 87 up osd.21
22 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 3.3 MiB
16 GiB 1.5 TiB 79.00 1.04 86 up osd.22
47 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.0 MiB
17 GiB 1.5 TiB 79.96 1.05 87 up osd.47
48 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 688 KiB
16 GiB 1.5 TiB 78.98 1.04 86 up osd.48
60 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.6 MiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.60
77 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.8 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.77
92 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.6 MiB
16 GiB 1.5 TiB 79.91 1.05 87 up osd.92
105 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.1 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.105
118 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.8 MiB
16 GiB 1.5 TiB 79.89 1.05 87 up osd.118
132 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 116 KiB
16 GiB 1.5 TiB 79.97 1.05 87 up osd.132
156 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 3.9 MiB
16 GiB 1.5 TiB 79.03 1.04 86 up osd.156
252 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.252
274 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.274
295 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.295
306 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.306
323 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.323
340 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.340
361 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.14 0.96 196 up osd.361
372 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.372
389 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.389
404 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 72.82 0.96 195 up osd.404
420 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.420
446 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.4 MiB
36 GiB 3.3 TiB 79.83 1.05 196 up osd.446
452 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.8 MiB
36 GiB 3.3 TiB 79.87 1.05 196 up osd.452
473 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.473
489 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.489
500 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.80 0.96 195 up osd.500
516 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 1.4 MiB
28 GiB 2.6 TiB 79.68 1.05 152 up osd.516
522 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 4.4 MiB
32 GiB 2.9 TiB 79.80 1.05 174 up osd.522
535 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 803 KiB
32 GiB 2.9 TiB 79.76 1.05 174 up osd.535
563 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.6 MiB
37 GiB 3.4 TiB 79.44 1.05 195 up osd.563
567 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 495 KiB
36 GiB 3.3 TiB 79.88 1.05 196 up osd.567
600 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.1 MiB
37 GiB 3.4 TiB 79.46 1.05 195 up osd.600
602 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.1 MiB
36 GiB 3.3 TiB 79.89 1.05 196 up osd.602
641 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.641
657 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.657
675 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.78 0.96 195 up osd.675
-25 566.02417 - 515 TiB 392 TiB 391 TiB 48 MiB
1.0 TiB 123 TiB 76.07 1.00 - host storage4-osd4
5 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 224 KiB
16 GiB 1.5 TiB 79.96 1.05 87 up osd.5
6 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.4 MiB
17 GiB 1.5 TiB 79.96 1.05 87 up osd.6
49 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 469 KiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.49
50 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.1 MiB
17 GiB 1.5 TiB 79.95 1.05 87 up osd.50
78 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 888 KiB
16 GiB 1.5 TiB 79.97 1.05 87 up osd.78
91 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.2 MiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.91
106 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.1 MiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.106
120 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 473 KiB
17 GiB 1.5 TiB 79.92 1.05 87 up osd.120
133 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 188 KiB
16 GiB 1.5 TiB 79.91 1.05 87 up osd.133
145 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.3 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.145
157 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.3 MiB
36 GiB 3.4 TiB 79.43 1.05 195 up osd.157
196 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.7 MiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.196
245 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.245
278 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.278
284 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.284
299 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.299
322 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.322
339 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.339
359 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.78 0.96 195 up osd.359
364 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.15 0.96 196 up osd.364
383 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.383
398 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.77 0.96 195 up osd.398
414 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.414
441 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.441
447 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.1 MiB
37 GiB 3.4 TiB 79.44 1.05 195 up osd.447
453 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.3 MiB
36 GiB 3.3 TiB 79.84 1.05 196 up osd.453
484 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.484
507 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 4.0 MiB
28 GiB 2.6 TiB 79.64 1.05 152 up osd.507
518 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 2.4 MiB
32 GiB 2.9 TiB 79.77 1.05 174 up osd.518
541 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 3.6 MiB
32 GiB 2.9 TiB 79.80 1.05 174 up osd.541
553 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.4 MiB
36 GiB 3.4 TiB 79.47 1.05 195 up osd.553
570 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.7 MiB
37 GiB 3.3 TiB 79.86 1.05 196 up osd.570
583 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.77 0.96 195 up osd.583
599 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 646 KiB
36 GiB 3.3 TiB 79.85 1.05 196 up osd.599
603 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.2 MiB
36 GiB 3.4 TiB 79.43 1.05 195 up osd.603
639 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.639
654 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.654
671 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.671
-16 566.02417 - 515 TiB 391 TiB 390 TiB 55 MiB
1.0 TiB 123 TiB 76.04 1.00 - host storage4-osd5
7 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 3.8 MiB
16 GiB 1.5 TiB 78.99 1.04 86 up osd.7
8 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 2.0 MiB
16 GiB 1.5 TiB 78.98 1.04 86 up osd.8
51 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.4 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.51
52 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.4 MiB
16 GiB 1.5 TiB 79.87 1.05 87 up osd.52
59 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 511 KiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.59
73 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.0 MiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.73
90 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 889 KiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.90
104 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.3 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.104
117 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 159 KiB
16 GiB 1.5 TiB 79.02 1.04 86 up osd.117
130 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.1 MiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.130
155 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 479 KiB
16 GiB 1.5 TiB 79.90 1.05 87 up osd.155
243 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.78 0.96 195 up osd.243
251 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 1.9 MiB
33 GiB 2.9 TiB 79.77 1.05 174 up osd.251
260 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.260
269 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.4 TiB 73.18 0.96 196 up osd.269
304 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.304
325 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.325
341 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.341
367 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.367
369 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.81 0.96 195 up osd.369
385 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.385
401 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.401
417 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.417
445 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.8 MiB
36 GiB 3.3 TiB 79.86 1.05 196 up osd.445
450 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.1 MiB
36 GiB 3.4 TiB 79.43 1.05 195 up osd.450
474 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.474
490 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.490
492 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.6 MiB
36 GiB 3.4 TiB 79.43 1.05 195 up osd.492
502 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.502
508 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 5.0 MiB
29 GiB 2.6 TiB 79.66 1.05 152 up osd.508
544 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 4.9 MiB
32 GiB 2.9 TiB 79.74 1.05 174 up osd.544
555 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.3 MiB
37 GiB 3.3 TiB 79.86 1.05 196 up osd.555
571 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.5 MiB
36 GiB 3.3 TiB 79.85 1.05 196 up osd.571
607 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.4 MiB
36 GiB 3.3 TiB 79.86 1.05 196 up osd.607
611 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.2 MiB
37 GiB 3.4 TiB 79.44 1.05 195 up osd.611
642 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.78 0.96 195 up osd.642
659 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.81 0.96 195 up osd.659
674 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.4 TiB 72.82 0.96 195 up osd.674
-28 566.02417 - 515 TiB 392 TiB 391 TiB 48 MiB
1.0 TiB 123 TiB 76.07 1.00 - host storage4-osd6
23 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 468 KiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.23
24 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.7 MiB
16 GiB 1.5 TiB 79.95 1.05 87 up osd.24
53 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 305 KiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.53
55 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.0 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.55
62 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.5 MiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.62
74 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.4 MiB
17 GiB 1.5 TiB 79.96 1.05 87 up osd.74
87 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.6 MiB
17 GiB 1.5 TiB 79.92 1.05 87 up osd.87
101 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.7 MiB
16 GiB 1.5 TiB 79.91 1.05 87 up osd.101
113 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 2.4 MiB
16 GiB 1.5 TiB 79.01 1.04 86 up osd.113
127 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 1.0 MiB
17 GiB 1.5 TiB 79.01 1.04 86 up osd.127
135 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 210 KiB
16 GiB 1.5 TiB 79.95 1.05 87 up osd.135
256 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.256
277 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.277
293 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.293
301 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.301
317 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.317
335 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.335
351 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.351
358 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.358
379 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.379
395 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.78 0.96 195 up osd.395
411 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.411
444 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 635 KiB
37 GiB 3.3 TiB 79.85 1.05 196 up osd.444
449 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 368 KiB
37 GiB 3.3 TiB 79.83 1.05 196 up osd.449
455 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.455
482 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.482
495 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.495
515 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 4.9 MiB
28 GiB 2.6 TiB 79.67 1.05 152 up osd.515
531 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 3.1 MiB
32 GiB 2.9 TiB 79.80 1.05 174 up osd.531
546 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 894 KiB
33 GiB 3.0 TiB 79.32 1.04 173 up osd.546
551 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.4 MiB
36 GiB 3.4 TiB 79.45 1.05 195 up osd.551
572 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.9 MiB
36 GiB 3.3 TiB 79.88 1.05 196 up osd.572
587 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.6 MiB
37 GiB 3.4 TiB 79.46 1.05 195 up osd.587
609 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.7 MiB
37 GiB 3.3 TiB 79.85 1.05 196 up osd.609
613 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.9 MiB
37 GiB 3.3 TiB 79.87 1.05 196 up osd.613
635 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.635
651 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.651
667 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.667
-31 566.02417 - 515 TiB 391 TiB 390 TiB 62 MiB
1.0 TiB 124 TiB 75.99 1.00 - host storage4-osd7
9 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.2 MiB
16 GiB 1.5 TiB 79.91 1.05 87 up osd.9
10 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 3.8 MiB
16 GiB 1.5 TiB 79.03 1.04 86 up osd.10
45 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.1 MiB
16 GiB 1.5 TiB 79.95 1.05 87 up osd.45
46 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.5 MiB
16 GiB 1.5 TiB 79.96 1.05 87 up osd.46
64 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 3.6 MiB
16 GiB 1.5 TiB 79.02 1.04 86 up osd.64
79 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.3 MiB
17 GiB 1.5 TiB 79.93 1.05 87 up osd.79
98 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 2.3 MiB
17 GiB 1.5 TiB 79.04 1.04 86 up osd.98
107 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.3 MiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.107
111 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 600 KiB
16 GiB 1.5 TiB 78.99 1.04 86 up osd.111
126 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.9 MiB
16 GiB 1.5 TiB 79.88 1.05 87 up osd.126
138 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 1.1 MiB
16 GiB 1.5 TiB 79.02 1.04 86 up osd.138
259 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.78 0.96 195 up osd.259
273 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.273
287 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.77 0.96 195 up osd.287
290 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.80 0.96 195 up osd.290
316 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.78 0.96 195 up osd.316
334 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.78 0.96 195 up osd.334
352 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.352
376 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.82 0.96 195 up osd.376
390 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.390
408 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.408
422 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.422
440 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.440
464 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.6 MiB
36 GiB 3.3 TiB 79.83 1.05 196 up osd.464
468 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.7 MiB
38 GiB 3.4 TiB 79.45 1.05 195 up osd.468
485 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.485
498 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.80 0.96 195 up osd.498
512 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 5.4 MiB
28 GiB 2.6 TiB 79.69 1.05 152 up osd.512
520 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 3.8 MiB
32 GiB 2.9 TiB 79.79 1.05 174 up osd.520
537 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 953 KiB
32 GiB 3.0 TiB 79.31 1.04 173 up osd.537
564 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.1 MiB
37 GiB 3.3 TiB 79.87 1.05 196 up osd.564
574 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 913 KiB
36 GiB 3.3 TiB 79.87 1.05 196 up osd.574
588 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.2 MiB
36 GiB 3.4 TiB 79.47 1.05 195 up osd.588
608 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.0 MiB
37 GiB 3.4 TiB 79.44 1.05 195 up osd.608
612 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.8 MiB
36 GiB 3.3 TiB 79.84 1.05 196 up osd.612
640 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.640
656 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.656
672 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.672
-34 566.02417 - 515 TiB 391 TiB 390 TiB 60 MiB
1.0 TiB 124 TiB 76.00 1.00 - host storage4-osd8
11 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.2 MiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.11
12 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.2 MiB
17 GiB 1.5 TiB 79.97 1.05 87 up osd.12
56 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 50 KiB
16 GiB 1.5 TiB 79.05 1.04 86 up osd.56
58 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 3.8 MiB
16 GiB 1.5 TiB 79.04 1.04 86 up osd.58
66 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 2.6 MiB
16 GiB 1.5 TiB 78.98 1.04 86 up osd.66
81 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.6 MiB
16 GiB 1.5 TiB 79.96 1.05 87 up osd.81
94 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 2.2 MiB
16 GiB 1.5 TiB 79.95 1.05 87 up osd.94
123 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.5 MiB
16 GiB 1.5 TiB 79.91 1.05 87 up osd.123
143 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 1.3 MiB
16 GiB 1.5 TiB 79.00 1.04 86 up osd.143
151 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.6 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.151
187 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.6 MiB
16 GiB 1.5 TiB 79.92 1.05 87 up osd.187
250 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.250
264 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.264
282 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.282
297 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.18 0.96 196 up osd.297
318 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.318
333 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.333
349 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.17 0.96 196 up osd.349
362 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.78 0.96 195 up osd.362
381 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.381
399 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.19 0.96 196 up osd.399
415 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.80 0.96 195 up osd.415
435 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.435
463 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.6 MiB
36 GiB 3.3 TiB 79.85 1.05 196 up osd.463
467 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.2 MiB
37 GiB 3.3 TiB 79.83 1.05 196 up osd.467
480 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.480
503 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 4.1 MiB
28 GiB 2.7 TiB 79.16 1.04 151 up osd.503
519 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 342 KiB
33 GiB 3.0 TiB 79.29 1.04 173 up osd.519
539 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 5.1 MiB
33 GiB 2.9 TiB 79.79 1.05 174 up osd.539
562 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.3 MiB
36 GiB 3.4 TiB 79.44 1.05 195 up osd.562
573 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.1 MiB
36 GiB 3.3 TiB 79.84 1.05 196 up osd.573
589 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.6 MiB
37 GiB 3.4 TiB 79.44 1.05 195 up osd.589
606 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.8 MiB
35 GiB 3.4 TiB 79.43 1.05 195 up osd.606
610 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 4.7 MiB
36 GiB 3.3 TiB 79.87 1.05 196 up osd.610
645 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.81 0.96 195 up osd.645
658 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.658
673 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.78 0.96 195 up osd.673
681 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.681
-37 566.02417 - 515 TiB 391 TiB 390 TiB 54 MiB
1.0 TiB 124 TiB 75.96 1.00 - host storage4-osd9
13 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.8 MiB
16 GiB 1.5 TiB 79.93 1.05 87 up osd.13
14 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 326 KiB
17 GiB 1.5 TiB 79.04 1.04 86 up osd.14
54 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 1.7 MiB
16 GiB 1.5 TiB 79.01 1.04 86 up osd.54
57 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.2 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.57
68 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 2.0 MiB
16 GiB 1.5 TiB 78.99 1.04 86 up osd.68
82 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 345 KiB
17 GiB 1.5 TiB 79.93 1.05 87 up osd.82
93 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 895 KiB
32 GiB 2.9 TiB 79.78 1.05 174 up osd.93
95 hdd 8.00156 1.00000 7.3 TiB 5.7 TiB 5.7 TiB 1.3 MiB
16 GiB 1.5 TiB 79.01 1.04 86 up osd.95
109 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.6 MiB
16 GiB 1.5 TiB 79.94 1.05 87 up osd.109
122 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 1.6 MiB
16 GiB 1.5 TiB 79.95 1.05 87 up osd.122
136 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.7 TiB 1.5 MiB
16 GiB 1.5 TiB 79.03 1.04 86 up osd.136
147 hdd 8.00156 1.00000 7.3 TiB 5.8 TiB 5.8 TiB 3.3 MiB
16 GiB 1.5 TiB 79.95 1.05 87 up osd.147
246 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.78 0.96 195 up osd.246
276 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.276
286 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.286
310 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.310
319 hdd 14.00052 1.00000 13 TiB 10 TiB 10 TiB 1.9 MiB
28 GiB 2.6 TiB 79.67 1.05 152 up osd.319
327 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 72.82 0.96 195 up osd.327
342 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 72.82 0.96 195 up osd.342
375 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.375
391 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.391
406 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 72.82 0.96 195 up osd.406
423 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.77 0.96 195 up osd.423
425 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 72.82 0.96 195 up osd.425
443 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 73.16 0.96 196 up osd.443
469 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 3.1 MiB
36 GiB 3.3 TiB 79.84 1.05 196 up osd.469
470 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.1 MiB
36 GiB 3.4 TiB 79.44 1.05 195 up osd.470
486 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.81 0.96 195 up osd.486
533 hdd 16.00090 1.00000 15 TiB 12 TiB 12 TiB 1.4 MiB
32 GiB 2.9 TiB 79.77 1.05 174 up osd.533
552 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.5 MiB
36 GiB 3.4 TiB 79.46 1.05 195 up osd.552
578 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 1.0 MiB
36 GiB 3.3 TiB 79.85 1.05 196 up osd.578
590 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 2.2 MiB
36 GiB 3.4 TiB 79.45 1.05 195 up osd.590
617 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.1 MiB
36 GiB 3.3 TiB 79.84 1.05 196 up osd.617
621 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB 5.1 MiB
36 GiB 3.3 TiB 79.85 1.05 196 up osd.621
631 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.79 0.96 195 up osd.631
646 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 31
GiB 4.5 TiB 72.79 0.96 195 up osd.646
663 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.5 TiB 72.80 0.96 195 up osd.663
678 hdd 18.00020 1.00000 16 TiB 12 TiB 12 TiB 0 B 32
GiB 4.4 TiB 72.83 0.96 195 up osd.678
TOTAL 8.0 PiB 6.1 PiB 6.1 PiB 44 GiB
17 TiB 1.9 PiB 75.99
MIN/MAX VAR: 0.07/1.05 STDDEV: 9.21
ceph osd crush rule dump:
[
{
"rule_id": 0,
"rule_name": "replicated_rule",
"ruleset": 0,
"type": 1,
"min_size": 1,
"max_size": 10,
"steps": [
{
"op": "take",
"item": -1,
"item_name": "default"
},
{
"op": "chooseleaf_firstn",
"num": 0,
"type": "host"
},
{
"op": "emit"
}
]
},
{
"rule_id": 1,
"rule_name": "ssdrule",
"ruleset": 1,
"type": 1,
"min_size": 1,
"max_size": 10,
"steps": [
{
"op": "take",
"item": -8,
"item_name": "ssd~ssd"
},
{
"op": "chooseleaf_firstn",
"num": 0,
"type": "host"
},
{
"op": "emit"
}
]
},
{
"rule_id": 2,
"rule_name": "cephfs_data",
"ruleset": 2,
"type": 3,
"min_size": 3,
"max_size": 12,
"steps": [
{
"op": "set_chooseleaf_tries",
"num": 5
},
{
"op": "set_choose_tries",
"num": 100
},
{
"op": "take",
"item": -1,
"item_name": "default"
},
{
"op": "chooseleaf_indep",
"num": 0,
"type": "host"
},
{
"op": "emit"
}
]
}
]
ceph osd pool ls detail:
pool 1 'device_health_metrics' replicated size 3 min_size 1 crush_rule 1
object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change
16984 flags hashpspool stripe_width 0 pg_num_min 1 application
mgr_devicehealth
pool 4 'cephfs_data' erasure profile ec_10_2 size 12 min_size 10
crush_rule 2 object_hash rjenkins pg_num 8192 pgp_num 8192
autoscale_mode warn last_change 1041923 lfor 0/132722/137197 flags
hashpspool,ec_overwrites stripe_width 40960 deep_scrub_interval 5e+06
scrub_max_interval 5.12e+07 application cephfs
pool 5 'cephfs_metadata' replicated size 3 min_size 1 crush_rule 1
object_hash rjenkins pg_num 128 pgp_num 128 autoscale_mode off
last_change 938674 lfor 0/20187/28908 flags hashpspool stripe_width 0
pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs
Thanks!
On 11/4/22 22:14, Joseph Mundackal wrote:
Hi Denis,
can you share the following data points?
ceph osd df tree (to see how the osd's are distributed)
ceph osd crush rule dump (to see what your ec rule looks like)
ceph osd pool ls detail (to see the pools and pools to crush rule
mapping and pg nums)
Also
"optimize_result": "Unable to find further optimization, or
pool(s) pg_num is decreasing, or distribution is already perfect
is the auto scaler currently adjusting your pg counts?
-Joseph
On Wed, Nov 2, 2022 at 5:01 PM Denis Polom <denispolom@xxxxxxxxx> wrote:
Hi Joseph,
thank you for answer. But if I'm looking correctly to 'ceph osd
df' output I posted I see there are about 195 PGs per OSD.
There are 608 OSDs in the pool, which is the only data pool. What
I have calculated - PG calc says that PG number is fine.
On 11/1/22 14:03, Joseph Mundackal wrote:
If the GB per pg is high, the balancer module won't be able to help.
Your pg count per osd also looks low (30's), so increasing pgs
per pool would help with both problems.
You can use the pg calculator to determine which pools need what
On Tue, Nov 1, 2022, 08:46 Denis Polom <denispolom@xxxxxxxxx> wrote:
Hi
I observed on my Ceph cluster running latest Pacific that
same size OSDs
are utilized differently even if balancer is running and
reports status
as perfectly balanced.
{
"active": true,
"last_optimize_duration": "0:00:00.622467",
"last_optimize_started": "Tue Nov 1 12:49:36 2022",
"mode": "upmap",
"optimize_result": "Unable to find further optimization,
or pool(s)
pg_num is decreasing, or distribution is already perfect",
"plans": []
}
balancer settings for upmap are:
mgr advanced
mgr/balancer/mode upmap
mgr advanced
mgr/balancer/upmap_max_deviation 1
mgr advanced
mgr/balancer/upmap_max_optimizations 20
It's obvious that utilization is not same (difference is
about 1TB) from
command `ceph osd df`. Following is just a partial output:
ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP
META AVAIL %USE VAR PGS STATUS
0 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
3.0 MiB
37 GiB 3.6 TiB 78.09 1.05 196 up
124 hdd 18.00020 1.00000 16 TiB 12 TiB 12
TiB 0 B 32
GiB 4.7 TiB 71.20 0.96 195 up
157 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
5.3 MiB 35
GiB 3.7 TiB 77.67 1.05 195 up
1 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
2.0 MiB
35 GiB 3.7 TiB 77.69 1.05 195 up
243 hdd 18.00020 1.00000 16 TiB 12 TiB 12
TiB 0 B 31
GiB 4.7 TiB 71.16 0.96 195 up
244 hdd 18.00020 1.00000 16 TiB 12 TiB 12
TiB 0 B 31
GiB 4.7 TiB 71.19 0.96 195 up
245 hdd 18.00020 1.00000 16 TiB 12 TiB 12
TiB 0 B 32
GiB 4.7 TiB 71.55 0.96 196 up
246 hdd 18.00020 1.00000 16 TiB 12 TiB 12
TiB 0 B 31
GiB 4.7 TiB 71.17 0.96 195 up
249 hdd 18.00020 1.00000 16 TiB 12 TiB 12
TiB 0 B 30
GiB 4.7 TiB 71.18 0.96 195 up
500 hdd 18.00020 1.00000 16 TiB 12 TiB 12
TiB 0 B 30
GiB 4.7 TiB 71.19 0.96 195 up
501 hdd 18.00020 1.00000 16 TiB 12 TiB 12
TiB 0 B 31
GiB 4.7 TiB 71.57 0.96 196 up
502 hdd 18.00020 1.00000 16 TiB 12 TiB 12
TiB 0 B 31
GiB 4.7 TiB 71.18 0.96 195 up
532 hdd 18.00020 1.00000 16 TiB 12 TiB 12
TiB 0 B 31
GiB 4.7 TiB 71.16 0.96 195 up
549 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
576 KiB 36
GiB 3.7 TiB 77.70 1.05 195 up
550 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
3.8 MiB 36
GiB 3.7 TiB 77.67 1.05 195 up
551 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
2.4 MiB 35
GiB 3.7 TiB 77.68 1.05 195 up
552 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
5.5 MiB 35
GiB 3.7 TiB 77.69 1.05 195 up
553 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
5.1 MiB 37
GiB 3.6 TiB 77.71 1.05 195 up
554 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
967 KiB 36
GiB 3.6 TiB 77.71 1.05 195 up
555 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
1.3 MiB 36
GiB 3.6 TiB 78.08 1.05 196 up
556 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
4.7 MiB 36
GiB 3.6 TiB 78.10 1.05 196 up
557 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
2.4 MiB 36
GiB 3.7 TiB 77.69 1.05 195 up
558 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
4.5 MiB 36
GiB 3.6 TiB 77.72 1.05 195 up
559 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
1.5 MiB 35
GiB 3.6 TiB 78.09 1.05 196 up
560 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
5.2 MiB 35
GiB 3.7 TiB 77.69 1.05 195 up
561 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
2.8 MiB 35
GiB 3.7 TiB 77.69 1.05 195 up
562 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
1.0 MiB 36
GiB 3.7 TiB 77.68 1.05 195 up
563 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
2.6 MiB 36
GiB 3.7 TiB 77.68 1.05 195 up
564 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
5.1 MiB 36
GiB 3.6 TiB 78.09 1.05 196 up
567 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
4.8 MiB 36
GiB 3.6 TiB 78.11 1.05 196 up
568 hdd 18.00020 1.00000 16 TiB 13 TiB 13 TiB
5.2 MiB 35
GiB 3.7 TiB 77.68 1.05 195 up
All OSDs are used by the same pool (EC)
I have the same issue on another Ceph cluster with the same
setup where
I was able to make OSDs utilization same by changing reweight
from
1.00000 to lower on OSDs with higher utilization and I got a
lot of
free space:
before changing reweight:
--- RAW STORAGE ---
CLASS SIZE AVAIL USED RAW USED %RAW USED
hdd 3.1 PiB 510 TiB 2.6 PiB 2.6 PiB 83.77
ssd 2.6 TiB 2.6 TiB 46 GiB 46 GiB 1.70
TOTAL 3.1 PiB 513 TiB 2.6 PiB 2.6 PiB 83.70
--- POOLS ---
POOL ID PGS STORED OBJECTS USED
%USED MAX AVAIL
cephfs_data 3 8192 2.1 PiB 555.63M 2.6 PiB
91.02 216 TiB
cephfs_metadata 4 128 7.5 GiB 140.22k 22 GiB
0.87 851 GiB
device_health_metrics 5 1 4.1 GiB 1.15k 8.3 GiB
0 130 TiB
after changing reweight:
--- RAW STORAGE ---
CLASS SIZE AVAIL USED RAW USED %RAW USED
hdd 3.1 PiB 522 TiB 2.6 PiB 2.6 PiB 83.38
ssd 2.6 TiB 2.6 TiB 63 GiB 63 GiB 2.36
TOTAL 3.1 PiB 525 TiB 2.6 PiB 2.6 PiB 83.31
--- POOLS ---
POOL ID PGS STORED OBJECTS USED
%USED MAX AVAIL
cephfs_data 3 8192 2.1 PiB 555.63M 2.5 PiB
86.83 330 TiB
cephfs_metadata 4 128 7.4 GiB 140.22k 22 GiB
0.87 846 GiB
device_health_metrics 5 1 4.2 GiB 1.15k 8.4 GiB
0 198 TiB
Free space I got is almost 5% what is about 100TB!
This is just workaround and I'm not happy with keeping
reweight with not
default value permanently.
Do you have any advice please, what settings can be adjusted
or should
be adjusted to keep OSDs utilization same? Because obviously
balancer
upmap, not even crush-compat are working correctly at least
in my case.
Many thanks!
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
