Circling back on this- this was caused by the ceph-insights mgr plugin. Disabled this and behavior stopped. This was seemingly bloating the mon database- for some reason each .sst file (about 200mb) contained only 5 paxos entries per file, so something in that plugin was stuffing a ton into those entries. Either way, that plugin seems pretty harmful. On Tue, Feb 18, 2020 at 5:38 PM Peter Woodman <peter@xxxxxxxxxxxx> wrote: > Yeah, applied that command. > > For some reason, after 3 days of this, the behavior calmed down, and the > size of the mon store shrank down to ~100MB, where previously it was > growing to upwards of 6GB. > > On Mon, Feb 17, 2020 at 3:14 AM Dan van der Ster <dan@xxxxxxxxxxxxxx> > wrote: > >> This means it has been applied: >> >> # ceph osd dump -f json | jq .require_osd_release >> "nautilus" >> >> -- dan >> >> On Mon, Feb 17, 2020 at 11:10 AM Marc Roos <M.Roos@xxxxxxxxxxxxxxxxx> >> wrote: >> > >> > >> > How do you check if you issued this command in the past? >> > >> > >> > -----Original Message----- >> > To: ceph-users@xxxxxxx >> > Subject: Re: Excessive write load on mons after upgrade >> > from 12.2.13 -> 14.2.7 >> > >> > Hi Peter, >> > >> > could be a totally different problem but did you run the command "ceph >> > osd require-osd-release nautilus" after the upgrade? >> > We had poor performance after upgrading to nautilus and running this >> > command fixed it. The same was reported by others for previous updates. >> > Here is my original message regarding this issue: >> > >> https://lists.ceph.io/hyperkitty/list/ceph-users@xxxxxxx/message/OYFRWSJXPVWGGVPXFIVFEVGPKTHRVWQH/ >> > >> > We did not observe the master election problem though. >> > >> > _______________________________________________ >> > ceph-users mailing list -- ceph-users@xxxxxxx >> > To unsubscribe send an email to ceph-users-leave@xxxxxxx >> _______________________________________________ >> ceph-users mailing list -- ceph-users@xxxxxxx >> To unsubscribe send an email to ceph-users-leave@xxxxxxx >> > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |