Hi all,
A similar question would be if it is possible to let passive mgr do the data collection!?
We run 14.2.6 on a medium 2.5PB cluster with over 900M objects (rbd and mainly S3) . At the moment, we face an issue with the prometheus exporter while it has high load. (e.g. while we insert a new host or a rebalance is active) The CPU usage of the active mgr daemon raises over 100% (this issue is still not completely solved, see discussion on ceph-users in "High CPU usage by ceph-mgr in 14.2.6") what leads to the fact that the daemon cannot answer the scrape request.
We already reduced the scrape frequency radically (I think it is at 30s and was already at 60s) with no effect. Also a curl on the active mgr doesn't respond. Once the load goes away the mgr can answer the requests again. But somehow, the monitoring is especially interesting while there is high load so this is not ideal. Did anybody experience a similar issue?
Thanks in advance,
Michael
On 27.02.20, 08:58, "黄明友" <hmy@v.photos> wrote:
thank you ,you are right.
------------------------------------------------------------------
发件人：Jan Fajerski <jfajerski@xxxxxxxx>
发送时间：2020年2月27日(星期四) 15:34
收件人：ceph-users <ceph-users@xxxxxxx>
主 题： Re: ceph prometheus module no export content
On Thu, Feb 27, 2020 at 10:20:07AM +0800, 黄明友 wrote:
> hi,all
> I had enable the prometheus module on my ceph cluster , the
> ceph version
> is ceph version 14.2.5 (ad5bd132e1492173c85fda2cc863152730b16a92) nauti
> lus (stable).
> when I eanble this modle, can get exported content from the prometheus
> module.
> but , when I restart all the ceph cluster's node , the module can't
> expor any content , curl http://host/metrics ; get blan;
> the tcp/9283 port is listen; I had disable and enable this module,but
> still no content export.
You're probably querying a standby mgr daemon. Check ceph -s for the current
active and query that one. For a prometheus server you always want to scrape all
mgr daemons. Also see https://docs.ceph.com/docs/master/mgr/prometheus/
> 黄明友
> IT基础架构部经理
> V.Photos 云摄影
> 移动电话: +86 13540630430
> 客服电话：400 - 806 - 5775
> 电子邮件: hmy@v.photos
> 官方网址: www.v.photos
> 上海 黄浦区中山东二路88号外滩SOHO3Q F栋 2层
> 北京 朝阳区光华路9号光华路SOHO二期南二门SOHO3Q 1层
> 广州 天河区林和中路136号天誉花园二期3Wcoffice 天誉青创社区
> 深圳 南山区蛇口网谷科技大厦二期A座102网谷双创街 1层
> 成都 成华区建设路世贸广场 7层
> [cid:__aliyun158277000719672821]
>_______________________________________________
>ceph-users mailing list -- ceph-users@xxxxxxx
>To unsubscribe send an email to ceph-users-leave@xxxxxxx
--
Jan Fajerski
Senior Software Engineer Enterprise Storage
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
