On Thu, Feb 14, 2019 at 11:57 AM Sage Weil <sweil@xxxxxxxxxx> wrote: > > On Thu, 14 Feb 2019, Travis Nielsen wrote: > > In Rook there are changes in progress to bring the configuration of > > daemons more inline with what Ceph expects. Rook in the past has done > > most everything through the ceph.conf. Now the movement is away from > > ceph.conf, to use command line parameters where possible and mon > > settings for everything else. This is based on the understanding that > > ceph.conf is less preferred going forward. This is based on > > conversations with Sage for the preferred configuration approach. > > > > As long as needed, Rook will still generate a config file where > > necessary, such as for c-v. The assumption was that c-v could take > > advantage of the config changes that other daemons are going through, > > or at least would move in the same direction. > > FWIW we want a similar shift in general. For example there is a new > command in nautilus, 'ceph config generate-minimal-conf', that spits out a > config that has just enough information to reliably enable the client to > talk to the mons to get everything else. > > I think the big question is: what pieces of information is ceph-volume > pulling out of ceph.conf? There are a few bits and pieces where ceph-volume reads from the config file: * osd_dmcrypt_key_size * mount flags for filestore OSDs * cluster fsid These aren't critical, except perhaps for the cluster fsid, which is used to properly identify OSDs that belong to the "current" cluster. In other tools like ceph-medic, it is merely poking around places to validate the state of nodes. If a node does not have a cluster FSID, who/what can tell if that is fine or if it is indeed a problem? > > sage > > > > > > > On Thu, Feb 14, 2019 at 5:34 AM Alfredo Deza <adeza@xxxxxxxxxx> wrote: > > > > > > Hi Travis, > > > > > > It seems there is a push towards being able to run Ceph tooling > > > without a configuration file (usually in /etc/ceph/ceph.conf) [0]. > > > > > > I believe the reference ticket may be coming from a rook need, do you > > > foresee rook being more of a "source of truth" for more operations in > > > a Ceph cluster? My concern is how other tools would be able to > > > interact with a Ceph cluster that may have some information in a > > > separate system like rook. > > > > > > In the case of inspecting clusters, tools like ceph-medic poke around > > > things that might be of interesting like ceph.conf. As you know, we > > > found an instance where a ceph-mgr pod in rook didn't define its > > > cluster fsid [1], and although that might not cause any issues, it was > > > able to detect a problem because that pod was different from the other > > > pods in the cluster. > > > > > > In short (and more towards Ceph in general) it would be good to > > > normalize the way clusters operate, so that tooling will not have to > > > try multiple sources for fetching information/configuration. Right now > > > we have some items in the Monitors, but we allow config files (and > > > multiple config files at that!) which is already tricky to get right. > > > > > > > > > -Alfredo > > > > > > [0] http://tracker.ceph.com/issues/38279 > > > [1] https://github.com/rook/rook/issues/2628 > > > >
