On Thu, Feb 14, 2019 at 04:57:08PM +0000, Sage Weil wrote:
On Thu, 14 Feb 2019, Travis Nielsen wrote:
In Rook there are changes in progress to bring the configuration of
daemons more inline with what Ceph expects. Rook in the past has done
most everything through the ceph.conf. Now the movement is away from
ceph.conf, to use command line parameters where possible and mon
settings for everything else. This is based on the understanding that
ceph.conf is less preferred going forward. This is based on
conversations with Sage for the preferred configuration approach.
As long as needed, Rook will still generate a config file where
necessary, such as for c-v. The assumption was that c-v could take
advantage of the config changes that other daemons are going through,
or at least would move in the same direction.
FWIW we want a similar shift in general. For example there is a new
command in nautilus, 'ceph config generate-minimal-conf', that spits out a
config that has just enough information to reliably enable the client to
talk to the mons to get everything else.
Fwiw I started looking into this in the context of
http://tracker.ceph.com/issues/38279. Adding cli flags for the bootstrap options
should be fairly straight forward.
I think adding 'ceph config' support will be a little more tricky. One thing I
was wondering about though is who c-v should assume to be. Say in c-v's config
parsing we call 'ceph config show <who>', but in some situations the <who> is
not clear.
In most cases <who> will probably an osd id, but very early in an 'lvm prepare'
there is no osd id yet. In those cases getting conf values from the mon is
probably not crucial, but there will have to be careful ordering to get those
values as soon as a <who> can be determined.
Happy to hear from others though.
I think the big question is: what pieces of information is ceph-volume
pulling out of ceph.conf?
sage
On Thu, Feb 14, 2019 at 5:34 AM Alfredo Deza <adeza@xxxxxxxxxx> wrote:
>
> Hi Travis,
>
> It seems there is a push towards being able to run Ceph tooling
> without a configuration file (usually in /etc/ceph/ceph.conf) [0].
>
> I believe the reference ticket may be coming from a rook need, do you
> foresee rook being more of a "source of truth" for more operations in
> a Ceph cluster? My concern is how other tools would be able to
> interact with a Ceph cluster that may have some information in a
> separate system like rook.
>
> In the case of inspecting clusters, tools like ceph-medic poke around
> things that might be of interesting like ceph.conf. As you know, we
> found an instance where a ceph-mgr pod in rook didn't define its
> cluster fsid [1], and although that might not cause any issues, it was
> able to detect a problem because that pod was different from the other
> pods in the cluster.
>
> In short (and more towards Ceph in general) it would be good to
> normalize the way clusters operate, so that tooling will not have to
> try multiple sources for fetching information/configuration. Right now
> we have some items in the Monitors, but we allow config files (and
> multiple config files at that!) which is already tricky to get right.
>
>
> -Alfredo
>
> [0] http://tracker.ceph.com/issues/38279
> [1] https://github.com/rook/rook/issues/2628
--
Jan Fajerski
Engineer Enterprise Storage
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton,
HRB 21284 (AG Nürnberg)
