Hi Sage, If weight set are created and updated, either offline via "crush optimize" (possible now) or via a ceph-mgr task (hopefully in the future), adding and removing OSDs won't work via the ceph cli (CrushWrapper errors out on create_or_move_item which is what osd crush create-or-move needs, for instance). Requiring a workflow where OSDs must be added to the crushmap instead of the usual ceph osd crush create-or-move is impractical. Instead, create_or_move_item should be modified to update the weight sets, if any. What we could not figure out a few weeks ago is which values make sense for a newly added OSD. Assuming the weight set are updated via an incremental rebalancing process, I think the weight set of a new OSD should simply be zero for all positions and the target weight is set as usual. The next time the rebalancing process runs, it will set the weight set to the right value and backfilling will start. Or, if it proceeds incrementally, it will gradually increase the weight set until it reaches the optimal value. From the user perspective, the only difference is that backfilling does not happen right away, it has to wait for the next rebalancing update. Preparing an OSD to be decomissionned can be done by setting the target weight to zero. The rebalancing process will (gradually or not) set the weight set to zero and all PGs will move out of the OSD. What do you think ? Cheers -- Loïc Dachary, Artisan Logiciel Libre -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
