Hello Stefan,
Thank you for your answer.
On Fri, Sep 2, 2022 at 5:27 PM Stefan Kooman <stefan@xxxxxx> wrote:
> On 9/2/22 15:55, Oebele Drijfhout wrote:
> > Hello,
> >
> > I'm new to Ceph and I recently inherited a 4 node cluster with 32 OSDs
> and
> > about 116TB raw space, which shows low available space, which I'm trying
> to
> > increase by enabling the balancer and lowering priority for the most-used
> > OSDs. My questions are: is what I did correct with the current state of
> the
> > cluster and can I do more to speed up rebalancing and will we actually
> make
> > more space available this way?
>
> Yes. When it's perfectly balanced the average OSD utilization should
> approach %RAW USED.
>
>
The variance between OSD utilization has been going down during the night,
but I'm worried that we will soon hit 95% full on osd.11. %USE is steadily
going up. What can I do to 1. prevent more data being written to this OSD
and 2. force data off this OSD?
[truiten@xxxxxxxxxx ~]$ sudo ceph --cluster eun osd df
ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL
%USE VAR PGS STATUS
0 hdd 3.63869 1.00000 3.6 TiB 1.4 TiB 1.4 TiB 907 MiB 3.3 GiB 2.2 TiB
39.56 0.67 127 up
1 hdd 3.63869 1.00000 3.6 TiB 1.1 TiB 1.1 TiB 581 MiB 2.5 GiB 2.6 TiB
29.61 0.50 126 up
2 hdd 3.63869 1.00000 3.6 TiB 2.2 TiB 2.2 TiB 701 MiB 4.2 GiB 1.5 TiB
59.56 1.01 125 up
3 hdd 3.63869 1.00000 3.6 TiB 2.5 TiB 2.5 TiB 672 MiB 5.5 GiB 1.1 TiB
68.95 1.16 131 up
4 hdd 3.63869 1.00000 3.6 TiB 2.5 TiB 2.5 TiB 524 MiB 5.7 GiB 1.1 TiB
68.88 1.16 116 up
5 hdd 3.63869 0.76984 3.6 TiB 2.7 TiB 2.7 TiB 901 MiB 4.9 GiB 976 GiB
73.81 1.25 105 up
6 hdd 3.63869 0.76984 3.6 TiB 2.7 TiB 2.7 TiB 473 MiB 5.0 GiB 972 GiB
73.90 1.25 99 up
7 hdd 3.63869 1.00000 3.6 TiB 1.8 TiB 1.8 TiB 647 MiB 3.5 GiB 1.8 TiB
49.27 0.83 125 up
8 hdd 3.63869 1.00000 3.6 TiB 1.6 TiB 1.6 TiB 624 MiB 3.1 GiB 2.0 TiB
44.21 0.75 124 up
9 hdd 3.63869 1.00000 3.6 TiB 2.4 TiB 2.4 TiB 934 MiB 4.8 GiB 1.3 TiB
64.76 1.09 121 up
10 hdd 3.63869 1.00000 3.6 TiB 2.2 TiB 2.1 TiB 525 MiB 4.0 GiB 1.5 TiB
59.12 1.00 127 up
11 hdd 3.63869 0.76984 3.6 TiB 3.4 TiB 3.4 TiB 431 MiB 6.2 GiB 239 GiB
93.59 1.58 84 up <---
12 hdd 3.63869 1.00000 3.6 TiB 2.1 TiB 2.1 TiB 777 MiB 4.2 GiB 1.5 TiB
59.02 1.00 124 up
13 hdd 3.63869 1.00000 3.6 TiB 1.4 TiB 1.4 TiB 738 MiB 3.2 GiB 2.2 TiB
39.46 0.67 125 up
14 hdd 3.63869 1.00000 3.6 TiB 2.2 TiB 2.1 TiB 560 MiB 6.2 GiB 1.5 TiB
59.11 1.00 122 up
15 hdd 3.63869 1.00000 3.6 TiB 2.1 TiB 2.1 TiB 575 MiB 4.4 GiB 1.5 TiB
59.06 1.00 123 up
16 hdd 3.63869 1.00000 3.6 TiB 1.8 TiB 1.8 TiB 625 MiB 3.4 GiB 1.8 TiB
49.24 0.83 124 up
17 hdd 3.63869 0.76984 3.6 TiB 2.7 TiB 2.7 TiB 696 MiB 5.1 GiB 958 GiB
74.28 1.25 93 up
18 hdd 3.63869 1.00000 3.6 TiB 2.0 TiB 2.0 TiB 210 MiB 3.6 GiB 1.6 TiB
54.94 0.93 125 up
19 hdd 3.63869 1.00000 3.6 TiB 2.1 TiB 2.1 TiB 504 MiB 5.2 GiB 1.5 TiB
59.08 1.00 124 up
20 hdd 3.63869 1.00000 3.6 TiB 2.5 TiB 2.5 TiB 796 MiB 5.0 GiB 1.1 TiB
69.14 1.17 121 up
21 hdd 3.63869 1.00000 3.6 TiB 1.6 TiB 1.6 TiB 679 MiB 3.6 GiB 2.0 TiB
44.25 0.75 123 up
22 hdd 3.63869 1.00000 3.6 TiB 2.5 TiB 2.5 TiB 682 MiB 4.9 GiB 1.1 TiB
68.86 1.16 125 up
23 hdd 3.63869 1.00000 3.6 TiB 1.6 TiB 1.6 TiB 575 MiB 3.1 GiB 2.0 TiB
44.83 0.76 124 up
24 hdd 3.63869 1.00000 3.6 TiB 2.1 TiB 2.1 TiB 517 MiB 3.9 GiB 1.5 TiB
59.00 1.00 125 up
25 hdd 3.63869 1.00000 3.6 TiB 2.2 TiB 2.1 TiB 836 MiB 4.5 GiB 1.5 TiB
59.12 1.00 121 up
26 hdd 3.63869 1.00000 3.6 TiB 2.5 TiB 2.5 TiB 520 MiB 5.0 GiB 1.1 TiB
69.31 1.17 109 up
27 hdd 3.63869 1.00000 3.6 TiB 2.0 TiB 2.0 TiB 861 MiB 3.8 GiB 1.7 TiB
54.13 0.91 126 up
28 hdd 3.63869 1.00000 3.6 TiB 1.8 TiB 1.8 TiB 256 MiB 4.3 GiB 1.8 TiB
49.21 0.83 122 up
29 hdd 3.63869 1.00000 3.6 TiB 2.7 TiB 2.7 TiB 998 MiB 5.1 GiB 980 GiB
73.69 1.24 126 up
30 hdd 3.63869 1.00000 3.6 TiB 2.2 TiB 2.1 TiB 1.1 GiB 4.1 GiB 1.5 TiB
59.16 1.00 123 up
31 hdd 3.63869 1.00000 3.6 TiB 2.3 TiB 2.3 TiB 689 MiB 4.4 GiB 1.3 TiB
64.02 1.08 123 up
TOTAL 116 TiB 69 TiB 69 TiB 21 GiB 140 GiB 48 TiB
59.19
MIN/MAX VAR: 0.50/1.58 STDDEV: 12.44
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
