Greetings. This has been alerting a fair bit: Nov 02 08:15:25 <zodbot> PROBLEM - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 5274 (noc01) Nov 02 08:24:46 <zodbot> ACKNOWLEDGEMENT - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 5274 (noc01) http://kevin.id.fedoraproject.org/ Nov 02 18:15:16 <zodbot> RECOVERY - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is OK: OK: fedmsg consumer FedoraBadgesConsumer backlog value is 4440 (noc01) Nov 02 21:17:19 <zodbot> PROBLEM - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 5715 (noc01) Nov 02 21:27:20 <zodbot> PROBLEM - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 5440 (noc01) Nov 02 22:07:19 <zodbot> RECOVERY - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is OK: OK: fedmsg consumer FedoraBadgesConsumer backlog value is 4207 (noc01) Nov 03 05:59:23 <zodbot> PROBLEM - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 5250 (noc01) Nov 03 06:03:17 <zodbot> ACKNOWLEDGEMENT - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 5250 (noc01) http://puiterwijk.id.fedoraproject.org/ Nov 03 07:49:19 <zodbot> RECOVERY - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is OK: OK: fedmsg consumer FedoraBadgesConsumer backlog value is 4964 (noc01) Nov 03 08:22:11 <zodbot> PROBLEM - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 5185 (noc01) Nov 03 11:53:24 <zodbot> PROBLEM - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 5621 (noc01) Nov 03 12:03:21 <zodbot> PROBLEM - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 5578 (noc01) Nov 03 13:17:07 <zodbot> PROBLEM - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 6395 (noc01) Nov 03 13:17:07 <zodbot> ACKNOWLEDGEMENT - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 6395 (noc01) http://kevin.id.fedoraproject.org/ Nov 03 22:43:19 <zodbot> RECOVERY - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is OK: OK: fedmsg consumer FedoraBadgesConsumer backlog value is 4574 (noc01) Nov 05 22:45:19 <zodbot> PROBLEM - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 5206 (noc01) Nov 05 22:55:21 <zodbot> PROBLEM - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 5834 (noc01) Nov 05 23:15:22 <zodbot> RECOVERY - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is OK: OK: fedmsg consumer FedoraBadgesConsumer backlog value is 0 (noc01) Nov 07 15:07:22 <zodbot> PROBLEM - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 5178 (noc01) Nov 07 15:27:19 <zodbot> PROBLEM - badges-backend01.phx2.fedoraproject.org/Check fedmsg-hub consumers backlog is WARNING: WARNING: fedmsg consumer FedoraBadgesConsumer backlog value is 5392 (noc01) +1s? I would apply the below and run a master -t nagios_client to make sure all clients are updated. From a2c969c4ed4d3f80f9307859807040222b3a4b25 Mon Sep 17 00:00:00 2001 From: Kevin Fenzi <kevin@xxxxxxxxx> Date: Mon, 7 Nov 2016 22:33:02 +0000 Subject: [PATCH] increase the badges backlog warning to 7000 to avoid constant alerts --- roles/nagios/client/templates/check_fedmsg_consumers.cfg.j2 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/roles/nagios/client/templates/check_fedmsg_consumers.cfg.j2 b/roles/nagios/client/templates/check_fedmsg_consumers.cfg.j2 index 2805772..0b4b973 100644 --- a/roles/nagios/client/templates/check_fedmsg_consumers.cfg.j2 +++ b/roles/nagios/client/templates/check_fedmsg_consumers.cfg.j2 @@ -47,7 +47,7 @@ command[check_fedmsg_cbacklog_app]={{libdir}}/nagios/plugins/check_fedmsg_consum command[check_fedmsg_cbacklog_value]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py fedmsg-irc IRCBotConsumer 10 50 command[check_fedmsg_cbacklog_pkgs]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py fedmsg-hub GenACLsConsumer 10 50 command[check_fedmsg_cbacklog_summershum]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py fedmsg-hub SummerShumConsumer 100 500 -command[check_fedmsg_cbacklog_badges_backend]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py fedmsg-hub FedoraBadgesConsumer 5000 10000 +command[check_fedmsg_cbacklog_badges_backend]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py fedmsg-hub FedoraBadgesConsumer 7000 10000 command[check_fedmsg_cbacklog_notifs_backend]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py fedmsg-hub FMNConsumer 15000 20000 command[check_fedmsg_cbacklog_bugzilla2fedmsg]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py moksha-hub BugzillaConsumer 10 100 command[check_fedmsg_cbacklog_fedimg_backend]={{libdir}}/nagios/plugins/check_fedmsg_consumer_backlog.py fedmsg-hub FedimgConsumer 2000 5000 -- 1.8.3.1
Attachment:
pgpeqxSgAqNvj.pgp
Description: OpenPGP digital signature
_______________________________________________ infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx
