Hi, Thank you for the prompt response. > Such a heterogenous setup is only possible with a large number of OSDs A heterogeneous setup is exactly what I am testing for. I wonder if the balancer will actually consider the WEIGHT when allocating the data to OSD. Let's say I have 3 hosts, with 10 OSDs in each host. 1st and 2nd host has an equivalent size, 10 TB each host, while 3rd host has 20TB. In this scenario what will happen? Is it still the same where the 1st and 2nd host will hit storage full while leaving 10TB in the 3rd host? If so, we should always try our best to balance the hosts to have similar storage space? Thanks for any advice. Isaiah ________________________________ From: Robert Sander <r.sander@xxxxxxxxxxxxxxxxxxx> Sent: Monday, December 19, 2022 3:42 PM To: ceph-users@xxxxxxx <ceph-users@xxxxxxx> Subject: Re: Ceph Reweight Only works in same host? Hi, Am 19.12.22 um 08:28 schrieb Isaiah Tang Yue Shun: > Currently, I have 3 OSDs in 3 different hosts: > > # ceph osd tree: > > ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF > -1 0.03908 root default > -3 0.00980 host node1 > 0 hdd 0.00980 osd.0 up 1.00000 1.00000 > -9 0 host node2 > -5 0.00980 host node3 > 1 hdd 0.00980 osd.1 up 1.00000 1.00000 > -7 0.01949 host node4 > 2 hdd 0.01949 osd.2 up 1.00000 1.00000 > > When I insert data into the pool, the data is equally distributed on 3 OSDs. (3 replicas) > However, I want the osd.2 to hold more data since it has more space. What additional data should OSD 2 hold? The pool has a size of 3, i.e. all data objects have three copies. The cluster only has 3 OSDs, so each OSD gets one copy. In this configuration your available capacity is that of the smallest OSD. You cannot use additional space in larger OSDs. Such a heterogenous setup is only possible with a large number of OSDs where the placement groups can be assigned more flexible to the OSDs. Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-43 Fax: 030 / 405051-19 Zwangsangaben lt. §35a GmbHG: HRB 220009 B / Amtsgericht Berlin-Charlottenburg, Geschäftsführer: Peer Heinlein -- Sitz: Berlin _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |