On 2020-10-25 05:33, Amudhan P wrote: > Yes, There is a unbalance in PG's assigned to OSD's. > `ceph osd df` output snip > ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META > AVAIL %USE VAR PGS STATUS > 0 hdd 5.45799 1.00000 5.5 TiB 3.6 TiB 3.6 TiB 9.7 MiB 4.6 > GiB 1.9 TiB 65.94 1.31 13 up > 1 hdd 5.45799 1.00000 5.5 TiB 1.0 TiB 1.0 TiB 4.4 MiB 1.3 > GiB 4.4 TiB 18.87 0.38 9 up > 2 hdd 5.45799 1.00000 5.5 TiB 1.5 TiB 1.5 TiB 4.0 MiB 1.9 > GiB 3.9 TiB 28.30 0.56 10 up > 3 hdd 5.45799 1.00000 5.5 TiB 2.1 TiB 2.1 TiB 7.7 MiB 2.7 > GiB 3.4 TiB 37.70 0.75 12 up > 4 hdd 5.45799 1.00000 5.5 TiB 4.1 TiB 4.1 TiB 5.8 MiB 5.2 > GiB 1.3 TiB 75.27 1.50 20 up > 5 hdd 5.45799 1.00000 5.5 TiB 5.1 TiB 5.1 TiB 5.9 MiB 6.7 > GiB 317 GiB 94.32 1.88 18 up > 6 hdd 5.45799 1.00000 5.5 TiB 1.5 TiB 1.5 TiB 5.2 MiB 2.0 > GiB 3.9 TiB 28.32 0.56 9 up > > MIN/MAX VAR: 0.19/1.88 STDDEV: 22.13 ceph balancer mode upmap ceph balancer on The balancer should start balancing and this should result in way more space available. Good to know that ceph df is based on the disk that is most full. There is all sorts of tuning available for the balancer, although I can't find it in the documentation. Ceph docu better project is working on that. See [1] for information. You can look up the python code to see what variables you can tune: /usr/share/ceph/mgr/balancer/module.py ceph config set mgr/balancer/begin_weekday 1 ceph config set mgr/balancer/end_weekday 5 ceph config set mgr mgr/balancer/begin_time 1000 ceph config set mgr mgr/balancer/end_time 1700 ^^ to restrict the balancer running only on weekdays (monday to friday) from 10:00 - 17:00 h. Gr. Stefan [1]: https://docs.ceph.com/en/latest/rados/operations/balancer/#balancer _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |