Oh, one more thing. fmn has been alerting a lot. I am pretty sure from your description this is because when it's rebuilding it's cache it doesn't process anything. Apr 19 00:20:39 <zodbot> Aurélien Bompard - ansible.git:ad66644567 [*roles/openshift-apps/fmn/templates/deploymentconfig.yml] FMN: run the IRC sender now Apr 19 00:40:56 <zodbot> PROBLEM - rabbitmq01.iad2.fedoraproject.org/Check queue fmn is CRITICAL: RABBITMQ_QUEUE CRITICAL - messages CRITICAL (830), messages_ready OK (805) messages_unacknowledged OK (25) consumers OK (1) (noc01) Apr 19 01:00:57 <zodbot> PROBLEM - rabbitmq01.iad2.fedoraproject.org/Check queue fmn is CRITICAL: RABBITMQ_QUEUE CRITICAL - messages CRITICAL (3052), messages_ready OK (3027) messages_unacknowledged OK (25) consumers OK (1) (noc01) Apr 19 02:00:57 <zodbot> PROBLEM - rabbitmq01.iad2.fedoraproject.org/Check queue fmn is CRITICAL: RABBITMQ_QUEUE CRITICAL - messages CRITICAL (9903), messages_ready OK (9878) messages_unacknowledged OK (25) consumers OK (1) (noc01) Apr 19 03:00:57 <zodbot> PROBLEM - rabbitmq01.iad2.fedoraproject.org/Check queue fmn is CRITICAL: RABBITMQ_QUEUE CRITICAL - messages CRITICAL (19733), messages_ready OK (19708) messages_unacknowledged OK (25) consumers OK (1) (noc01) Apr 19 04:00:57 <zodbot> PROBLEM - rabbitmq01.iad2.fedoraproject.org/Check queue fmn is CRITICAL: RABBITMQ_QUEUE CRITICAL - messages CRITICAL (28472), messages_ready OK (28447) messages_unacknowledged OK (25) consumers OK (1) (noc01) Apr 19 05:00:57 <zodbot> PROBLEM - rabbitmq01.iad2.fedoraproject.org/Check queue fmn is CRITICAL: RABBITMQ_QUEUE CRITICAL - messages CRITICAL (36369), messages_ready OK (36344) messages_unacknowledged OK (25) consumers OK (1) (noc01) Apr 19 05:40:57 <zodbot> RECOVERY - rabbitmq01.iad2.fedoraproject.org/Check queue fmn is OK: RABBITMQ_QUEUE OK - messages OK (1) messages_ready OK (0) messages_unacknowledged OK (1) consumers OK (1) All queues under the thresholds (noc01) Apr 19 13:42:58 <zodbot> PROBLEM - rabbitmq01.iad2.fedoraproject.org/Check queue fmn is WARNING: RABBITMQ_QUEUE WARNING - messages WARNING (282), messages_ready OK (257) messages_unacknowledged OK (25) consumers OK (1) (noc01) Apr 19 13:52:58 <zodbot> PROBLEM - rabbitmq01.iad2.fedoraproject.org/Check queue fmn is CRITICAL: RABBITMQ_QUEUE CRITICAL - messages CRITICAL (1257), messages_ready OK (1232) messages_unacknowledged OK (25) consumers OK (1) (noc01) So, we should figure out a way to not do this. ;) We could just bump the check up to like 40000 or something? Or you all could revisit just continuing to process while rebuilding the cache? Or something. :) kevin
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ infrastructure mailing list -- infrastructure@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe send an email to infrastructure-leave@xxxxxxxxxxxxxxxxxxxxxxx Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@xxxxxxxxxxxxxxxxxxxxxxx Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
