Mandi! sinan@xxxxxxxx In chel di` si favelave... > What I don't get is, when I perform 'ceph osd out <osd>' the cluster is > rebalancing, but when I perform 'ceph osd crush remove osd.<osd>' it again > starts to rebalance. Why does this happen? I've recently hit the same 'strangeness'. Note that i'm not a ceph developer or 'power' (or 'old') user. Seems to me that there's two ''rebalance'': one for safety, one for optimization. If you tear 'out' an OSD, ceph rebalance the data for safety. But you don't have touched the crushmap, so data are scattered with the 'old' crushmap. So if then you remove that OSD (or, in any other way you touch the crushmap), a rebalance for 'optimization' start. In the same way, you can put 'slowly out' an OSD with: ceph osd reweight <ID> X (with 0 <= X <= 1) but still you don't touch the crusmap. You can also 'slowly remove' an OSD with: ceph osd crush reweight osd.<ID> X (with 0 <= X <= <disk_size_in_TB>); in this way you can 'deweight' the OSD in crushmap until 0, then you can safely remove. I hope i've not sayed too much blasphemy... ;-) -- dott. Marco Gaiarin GNUPG Key ID: 240A3D66 Associazione ``La Nostra Famiglia'' http://www.lanostrafamiglia.it/ Polo FVG - Via della Bontà, 7 - 33078 - San Vito al Tagliamento (PN) marco.gaiarin(at)lanostrafamiglia.it t +39-0434-842711 f +39-0434-842797 Dona il 5 PER MILLE a LA NOSTRA FAMIGLIA! http://www.lanostrafamiglia.it/index.php/it/sostienici/5x1000 (cf 00307430132, categoria ONLUS oppure RICERCA SANITARIA) _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com