Re: Extra daemons/servers reporting to mgr

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Jun 14, 2017 at 12:50 PM, Sage Weil <sweil@xxxxxxxxxx> wrote:
> On Wed, 14 Jun 2017, Yehuda Sadeh-Weinraub wrote:
>> On Sun, Jun 11, 2017 at 5:04 AM, John Spray <jspray@xxxxxxxxxx> wrote:
>> > MgrClient instances (such as those in the daemons, and those in every
>> > librados instance) open a session with ceph-mgr where they identify
>> > themselves by entity name and type.  ceph-mgr sends a MgrConfigure
>> > message, which tells the client whether to both sending stats and how
>> > often.  ceph-mgr also keeps a copy of the metadata (a la "ceph osd
>> > metadata...") for OSD and MDS daemons -- it loads all that up at
>> > startup, and then also freshens it when it sees a daemon restart or
>> > sees a new daemon.
>> >
>> > We would like to have something similar so that the mgr can be aware
>> > of the existence of other services like RGW gateways, RBD mirror
>> > services, perhaps also NFS gateways.
>> >
>> > The information about each daemon would at a minimum be its identity,
>> > type, and some static metadata.  It might also include some dynamic
>> > state/health structure.  The challenging part here is how to expose
>> > that to the various daemons, given that things like RGW are not known
>> > in advance to core Ceph and that they just consume the librados
>> > interface.
>>
>>
>> To start with, for radosgw process we could provide:
>
> It seems like these fall into two categories:
>
> 1/ static information about the daemon instance
>
>>  - realm name (+ id)
>>  - zonegroup name (+ id)
>>  - zone name (+ id )
>>  - instance id # (changes between runs)
>>  - listening ip:port
>>  - list of domains handled (maybe? can be quite large)
>>  - max number of concurrent requests
>
> This stuff would populate a 'running rgw daemons' type view, and could be
> used by an agent that dynamically configures a load balancer (haproxy or
> whatever).
>
> 2/ dynamic values that can be exposed as perfcounters
>
>>  - current period
>>  - current # of requests (or max # of requests in the past X minutes)
>  - current bandwidth
>  - etc
>
> Is there anything that is updated at runtime (not startup) that
> doesn't fit into a perfcounter?  It not, then a single call to something
> like rados_register_daemon(name, metadata_map) (and some mechanism
> for cleaning out dead people) ought to suffice?
>
>> There is some other info that falls under different aspects of the rgw
>> cluster, e.g., all the realm info / structure, the zonegroup config,
>> zone config. This kind of data should be a category of its own, and
>> more of a container to the specific radosgw process info. Does it make
>> sense in the ceph-mgr context?
>
> This stuff any potential mgr module can pull out of the zone metadata pool
> itself to display, right?  rgw doesn't need to translate it and
> report it?

Right, that's another option.

>
>> > It doesn't feel like a particularly natural thing for librados, but
>> > ultimately whatever we expose to rgw/rbd is de-facto librados, even if
>> > we put it in a different library or whatever.
>> >
>> > So far I've got as far as thinking we should have an extra call just
>> > in the C++ bindings that lets callers say "Hi, I'm a service not just
>> > a client, and here's a map of metadata", that they call one time
>> > between creating their RadosClient and connecting to the cluster.
>> >
>>
>>
>> It'd be great if the client itself could provide a flexible schema
>> mapping of the info that it needs to expose. Or if there was some
>> other generic way to do it. Was thinking something like the way you
>> send a json to elasticsearch and it generates a doc out of it, but
>> there's also a way to create a fixed mapping for the stored data.
>
> Hmm, instead of a map<string,string> it could just be a json blob that
> the mgr stores that it is up to the user to parse and interpret.  Or, if
> there is info that doesn't fit into a simple dict, then one dict item
> ("extra_stuff") can have a value consisting of encoded json.
>

Providing a json is probably easy and generic enough.

Yehuda
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux