Hi, Having CRUSH updates on a OSD start is something which is very useful. Tools like Ansible, Puppet and Salt can provision ceph.conf or other scripts which can be run as hooks to inject OSDs as the right location in the CRUSHMap. Something that is lacking (imho) is a 'commit and confirm' mode. Right now, after you deploy a new OSD with ceph-volume it's created as an new OSD and also injected into the CRUSHMap. Topology changes right away and backfills start. In certain scenarios it would be great if these OSDs would be added to the OSDMap and the CRUSH changes are staged in the MONs, but not committed yet. This way the OSDs start to talk with the MONs and you can perform some tests on them. The Mgr daemons start to collect data from them (although they are idle). An idea would be that you can set the MONs in 'CRUSH commit mode' like: $ ceph osd crush commit The cluster now goes into WARN mode and all changes to the CRUSHMap are staged, but not live yet. $ ceph osd crush diff This will show you the changes between the active CRUSHMap and the changes which are staged. Once you finished deploying your OSDs, testing everything you can run: $ ceph osd crush confirm Or, if you think the changes should be discarded: $ ceph osd crush discard After you 'confirm' the changes a new CRUSHMap is generated by the Monitors and send out to the cluster. This also prevents creating a new OSDMap every time an OSD is added. Adding 200 OSDs would generate one CRUSH change instead of 200 small ones. Does this idea sound like a sane idea? Thanks! Wido
