Hi Ashley, Thanks for your response. Nothing that I can think of would have happened. We are using max_mds =1. We do have 4 so used to have 3 standby. Within minutes they all crash. On Fri, May 1, 2020 at 2:21 PM Ashley Merrick <singapore@xxxxxxxxxxxxxx> wrote: > Quickly checking the code that calls that assert > > > > > if (version > omap_version) { > > omap_version = version; > > omap_num_objs = num_objs; > > omap_num_items.resize(omap_num_objs); > > journal_state = jstate; > > } else if (version == omap_version) { > > ceph_assert(omap_num_objs == num_objs); > > if (jstate > journal_state) > > journal_state = jstate; > > } > > } > > > Im not a dev, but not sure if this will help, seems could mean that MDS > thinks its behind on omaps/too far ahead. > > > Anything happened recently? Just running a single MDS? > > > Hopefully someone else may see this and shine some light on what could be > causing it. > > > > ---- On Sat, 02 May 2020 02:10:58 +0800 marcopizzolo@xxxxxxxxx wrote ---- > > > Hello, > > Hoping you can help me. > > Ceph had been largely problem free for us for the better part of a year. > We have a high file count in a single CephFS filesystem, and are seeing > this error in the logs: > > > /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.9/rpm/el7/BUILD/ceph-14.2.9/src/mds/OpenFileTable.cc: > 777: FAILED ceph_assert(omap_num_objs == num_objs) > > The issued seemed to occur this morning, and restarting the MDS as well as > rebooting the servers doesn't correct the problem. > > Not really sure where to look next as the MDS daemons crash. > > Appreciate any help you can provide > > Marco > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx