Hi Dan and Gregory, thanks! These are good pointers. Will look into that tomorrow. Best regards, ================= Frank Schilder AIT Risø Campus Bygning 109, rum S14 ________________________________________ From: Gregory Farnum <gfarnum@xxxxxxxxxx> Sent: 09 February 2023 17:12:23 To: Dan van der Ster Cc: Frank Schilder; ceph-users@xxxxxxx Subject: Re: Re: Frequent calling monitor election Also, that the current leader (ceph-01) is one of the monitors proposing an election each time suggests the problem is with getting commit acks back from one of its followers. On Thu, Feb 9, 2023 at 8:09 AM Dan van der Ster <dvanders@xxxxxxxxx> wrote: > > Hi Frank, > > Check the mon logs with some increased debug levels to find out what > the leader is busy with. > We have a similar issue (though, daily) and it turned out to be > related to the mon leader timing out doing a SMART check. > See https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftracker.ceph.com%2Fissues%2F54313&data=05%7C01%7Cfrans%40dtu.dk%7C46a1aa2c3396405cf7c208db0ab8778f%7Cf251f123c9ce448e927734bb285911d9%7C0%7C0%7C638115559618308316%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3kX6gzK%2FPvpVsWZTZ%2BkrzqwjAyLJ1XFKSggiuvvbRNc%3D&reserved=0 for how I debugged that. > > Cheers, Dan > > On Thu, Feb 9, 2023 at 7:56 AM Frank Schilder <frans@xxxxxx> wrote: > > > > Hi all, > > > > our monitors have enjoyed democracy since the beginning. However, I don't share a sudden excitement about voting: > > > > 2/9/23 4:42:30 PM[INF]overall HEALTH_OK > > 2/9/23 4:42:30 PM[INF]mon.ceph-01 is new leader, mons ceph-01,ceph-02,ceph-03,ceph-25,ceph-26 in quorum (ranks 0,1,2,3,4) > > 2/9/23 4:42:26 PM[INF]mon.ceph-01 calling monitor election > > 2/9/23 4:42:26 PM[INF]mon.ceph-26 calling monitor election > > 2/9/23 4:42:26 PM[INF]mon.ceph-25 calling monitor election > > 2/9/23 4:42:26 PM[INF]mon.ceph-02 calling monitor election > > 2/9/23 4:40:00 PM[INF]overall HEALTH_OK > > 2/9/23 4:30:00 PM[INF]overall HEALTH_OK > > 2/9/23 4:24:34 PM[INF]overall HEALTH_OK > > 2/9/23 4:24:34 PM[INF]mon.ceph-01 is new leader, mons ceph-01,ceph-02,ceph-03,ceph-25,ceph-26 in quorum (ranks 0,1,2,3,4) > > 2/9/23 4:24:29 PM[INF]mon.ceph-01 calling monitor election > > 2/9/23 4:24:29 PM[INF]mon.ceph-02 calling monitor election > > 2/9/23 4:24:29 PM[INF]mon.ceph-03 calling monitor election > > 2/9/23 4:24:29 PM[INF]mon.ceph-01 calling monitor election > > 2/9/23 4:24:29 PM[INF]mon.ceph-26 calling monitor election > > 2/9/23 4:24:29 PM[INF]mon.ceph-25 calling monitor election > > 2/9/23 4:24:29 PM[INF]mon.ceph-02 calling monitor election > > 2/9/23 4:24:04 PM[INF]overall HEALTH_OK > > 2/9/23 4:24:03 PM[INF]mon.ceph-01 is new leader, mons ceph-01,ceph-02,ceph-03,ceph-25,ceph-26 in quorum (ranks 0,1,2,3,4) > > 2/9/23 4:23:59 PM[INF]mon.ceph-01 calling monitor election > > 2/9/23 4:23:59 PM[INF]mon.ceph-02 calling monitor election > > 2/9/23 4:20:00 PM[INF]overall HEALTH_OK > > 2/9/23 4:10:00 PM[INF]overall HEALTH_OK > > 2/9/23 4:00:00 PM[INF]overall HEALTH_OK > > 2/9/23 3:50:00 PM[INF]overall HEALTH_OK > > 2/9/23 3:43:13 PM[INF]overall HEALTH_OK > > 2/9/23 3:43:13 PM[INF]mon.ceph-01 is new leader, mons ceph-01,ceph-02,ceph-03,ceph-25,ceph-26 in quorum (ranks 0,1,2,3,4) > > 2/9/23 3:43:08 PM[INF]mon.ceph-01 calling monitor election > > 2/9/23 3:43:08 PM[INF]mon.ceph-26 calling monitor election > > 2/9/23 3:43:08 PM[INF]mon.ceph-25 calling monitor election > > > > We moved a switch from one rack to another and after the switch came beck up, the monitors frequently bitch about who is the alpha. How do I get them to focus more on their daily duties again? > > > > Thanks for any help! > > ================= > > Frank Schilder > > AIT Risø Campus > > Bygning 109, rum S14 > > _______________________________________________ > > ceph-users mailing list -- ceph-users@xxxxxxx > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |