On Tue, 31 Jul 2012, Tommi Virtanen wrote: > On Tue, Jul 24, 2012 at 6:15 AM, <loic.dachary@xxxxxxxxxxxx> wrote: > > Note that if puppet client was run on nodeB before it was run on nodeA, all > > three steps would have been run in sequence instead of being spread over two > > puppet client invocations. > > Unfortunately, even that is not enough. > > The relevant keys (client.admin, client.bootstrap-osd, later > bootstrap-mds radosgw etc also) can only be created once the mons have > reached quorum. This is some time after they have started, even in the > best case. Making the puppet/chef run wait for that sounds like a bad > idea; especially since I use further chef-client runs to feed ceph-mon > information about its peers, which may be necessary for it to ever > reach quorum. > > While I can find ways of making the key generation happen as soon as > quorum is reached, communicating the keys to other nodes only happens > at the mercy of the configuration management system; both puppet and > chef seem to be in the mindset of "run every N minutes" option. So > even if we generate the keys best case 2 seconds after ceph-mon > startup, it needs a full configuration manager run on the source node, > and then a run on the destination node, before OSD bring-up etc can > succeed. > > I have found no satisfying solution to this, so far. It is also possible to feed initial keys to the monitors during the 'mkfs' stage. If the keys can be agreed on somehow beforehand, then they will already be in place when the initial quorum is reached. Not sure if that helps in this situation or not... sage -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
