I found the problem, thanks. There is a tracker ticket: https://tracker.ceph.com/issues/52820 On Fri, Oct 8, 2021 at 8:01 AM Jonathan D. Proulx <jon@xxxxxxxxxxxxx> wrote: > > Hi Patrick, > > Yes we had been successfully running on Pacific v16.2.5 > > Thanks for the pointer to the bug, we eventually ended up taking > eveything down and rebuilding the monstore using > monstore-tool. Perhaps a longer and less pleasant path than necessary > but it was effective. > > -Jon > > On Thu, Oct 07, 2021 at 09:11:21PM -0400, Patrick Donnelly wrote: > :Hello Jonathan, > : > :On Tue, Oct 5, 2021 at 9:13 AM Jonathan D. Proulx <jon@xxxxxxxxxxxxx> wrote: > :> > :> In the middle of a normal cephadm upgrade from 16.2.5 to 16.2.6, after the mgrs had successfully upgraded, 2/5 mons didn’t come back up (and the upgrade stopped at that point). Attempting to manually restart the crashed mons resulted in **all** of the other mons crashing too, usually with: > :> > :> terminate called after throwing an instance of 'ceph::buffer::v15_2_0::malformed_input' what(): void FSMap::decode(ceph::buffer::v15_2_0::list::const_iterator&) no longer understand old encoding version v < 7: Malformed input > : > :You upgraded from v16.2.5 and not Octopus? I would expect your cluster > :to crash when upgrading to any version of Pacific: > : > :https://tracker.ceph.com/issues/51673 > : > :Only the crash error has changed from an assertion to an exception. > : > :-- > :Patrick Donnelly, Ph.D. > :He / Him / His > :Principal Software Engineer > :Red Hat Sunnyvale, CA > :GPG: 19F28A586F808C2402351B93C3301A3E258DD79D > : > > -- > -- Patrick Donnelly, Ph.D. He / Him / His Principal Software Engineer Red Hat Sunnyvale, CA GPG: 19F28A586F808C2402351B93C3301A3E258DD79D _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx