On Mon, Jun 12, 2017 at 1:33 PM, John Spray <jspray@xxxxxxxxxx> wrote: > On Mon, Jun 12, 2017 at 2:09 PM, Casey Bodley <cbodley@xxxxxxxxxx> wrote: >> >> On 06/11/2017 08:04 AM, John Spray wrote: >>> >>> MgrClient instances (such as those in the daemons, and those in every >>> librados instance) open a session with ceph-mgr where they identify >>> themselves by entity name and type. ceph-mgr sends a MgrConfigure >>> message, which tells the client whether to both sending stats and how >>> often. ceph-mgr also keeps a copy of the metadata (a la "ceph osd >>> metadata...") for OSD and MDS daemons -- it loads all that up at >>> startup, and then also freshens it when it sees a daemon restart or >>> sees a new daemon. >>> >>> We would like to have something similar so that the mgr can be aware >>> of the existence of other services like RGW gateways, RBD mirror >>> services, perhaps also NFS gateways. >>> >>> The information about each daemon would at a minimum be its identity, >>> type, and some static metadata. It might also include some dynamic >>> state/health structure. The challenging part here is how to expose >>> that to the various daemons, given that things like RGW are not known >>> in advance to core Ceph and that they just consume the librados >>> interface. >>> >>> It doesn't feel like a particularly natural thing for librados, but >>> ultimately whatever we expose to rgw/rbd is de-facto librados, even if >>> we put it in a different library or whatever. >>> >>> So far I've got as far as thinking we should have an extra call just >>> in the C++ bindings that lets callers say "Hi, I'm a service not just >>> a client, and here's a map of metadata", that they call one time >>> between creating their RadosClient and connecting to the cluster. >>> >>> John >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> >> Hi John, >> >> Is there a reason that radosgw can't instantiate its own MgrClient instance >> and report status through that, instead of having to go through librados? If >> we have to report an entity instance, we could choose from one of our Rados >> clients. > > Currently librados does internally have a MgrClient already, so I > think I was instinctively looking to reuse that. > > However, the MgrClient in librados is only there so that it can issue > commands (i.e. for use in CLI), so one option would be to disable that > MgrClient by default (unless someone is actually calling > mgr_command()), and instantiate an external one. > > For things like NFS-ganesha though, we can't use MgrClient unless we > formalize it as an external API... Perhaps somebody can draw up what data we actually want to share from NFS-Ganesha to the manager? (Or from RGW?) It's not unreasonable to let RGW set up its own MgrClient, since it's an in-tree service and can be kept more tightly bound than external librados users. But if we can make a simple interface that serves both needs, that would be better. Maybe something like void share_metadata(map<string,string>); that can be invoked whenever desired, and the MgrClient sends that in whenever it's doing a report? Or maybe we want something that more explicitly maps from static metadata (like kind and version info) versus perfcounter-like state. But I'm not sure we actually want to restrict shared data to going in via services. It seems perfectly reasonable to collect IO and other statistics from cooperating clients, if we can do that without overwhelming the manager service. -Greg -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
