|
Hello, I currently have 5 servers within my CEPH Cluster 2 x (10 * 8TB Disks) 3 x (10 * 4TB Disks) Currently seeing a larger difference in OSD use across the two separate server types, as well as within the server itself. For example on one 4TB server I have an OSD at 64% and one at 84%, where on the 8TB servers the OSD range from 49% to 64%, where the highest used OSD’s are on the 4TB. Each drive has a weight set correctly for the drive size and each server has the correct weight set, below is my crush map. Apart from running the command to adjust the re-weight is there anything I am doing wrong or should change for better
spread of data, not looking for near perfect but where the 8TB drives are sitting at 64% max and 4TB are sitting at 80%’s causes a big inbalance. # begin crush map tunable choose_local_tries 0 tunable choose_local_fallback_tries 0 tunable choose_total_tries 50 tunable chooseleaf_descend_once 1 tunable chooseleaf_vary_r 1 tunable straw_calc_version 1 tunable allowed_bucket_algs 54 # buckets host sn1 { id -2 # do not change unnecessarily # weight 72.800 alg straw2 hash 0 # rjenkins1 item osd.0 weight 7.280 item osd.1 weight 7.280 item osd.3 weight 7.280 item osd.4 weight 7.280 item osd.2 weight 7.280 item osd.5 weight 7.280 item osd.6 weight 7.280 item osd.7 weight 7.280 item osd.8 weight 7.280 item osd.9 weight 7.280 } host sn3 { id -6 # do not change unnecessarily # weight 72.800 alg straw2 hash 0 # rjenkins1 item osd.10 weight 7.280 item osd.11 weight 7.280 item osd.12 weight 7.280 item osd.13 weight 7.280 item osd.14 weight 7.280 item osd.15 weight 7.280 item osd.16 weight 7.280 item osd.17 weight 7.280 item osd.18 weight 7.280 item osd.19 weight 7.280 } host sn4 { id -7 # do not change unnecessarily # weight 36.060 alg straw2 hash 0 # rjenkins1 item osd.20 weight 3.640 item osd.21 weight 3.640 item osd.22 weight 3.640 item osd.23 weight 3.640 item osd.24 weight 3.640 item osd.25 weight 3.640 item osd.26 weight 3.640 item osd.27 weight 3.640 item osd.28 weight 3.640 item osd.29 weight 3.300 } host sn5 { id -8 # do not change unnecessarily # weight 36.060 alg straw2 hash 0 # rjenkins1 item osd.30 weight 3.640 item osd.31 weight 3.640 item osd.32 weight 3.640 item osd.33 weight 3.640 item osd.34 weight 3.640 item osd.35 weight 3.640 item osd.36 weight 3.640 item osd.37 weight 3.640 item osd.38 weight 3.640 item osd.39 weight 3.640 } host sn6 { id -9 # do not change unnecessarily # weight 36.060 alg straw2 hash 0 # rjenkins1 item osd.40 weight 3.640 item osd.41 weight 3.640 item osd.42 weight 3.640 item osd.43 weight 3.640 item osd.44 weight 3.640 item osd.45 weight 3.640 item osd.46 weight 3.640 item osd.47 weight 3.640 item osd.48 weight 3.640 item osd.49 weight 3.640 } root default { id -1 # do not change unnecessarily # weight 253.780 alg straw2 hash 0 # rjenkins1 item sn1 weight 72.800 item sn3 weight 72.800 item sn4 weight 36.060 item sn5 weight 36.060 item sn6 weight 36.060 } # rules rule replicated_ruleset { ruleset 0 type replicated min_size 1 max_size 10 step take default step chooseleaf firstn 0 type host step emit } Thanks, Ashley |
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
