> Op 3 nov. 2016 om 19:13 heeft Joao Eduardo Luis <joao@xxxxxxx> het volgende geschreven: > >> On 11/03/2016 05:52 PM, wido@xxxxxxxx wrote: >> >> >>>> Op 3 nov. 2016 om 16:44 heeft Joao Eduardo Luis <joao@xxxxxxx> het volgende geschreven: >>>> >>>>> On 11/03/2016 01:24 PM, Wido den Hollander wrote: >>>>> >>>>> Op 3 november 2016 om 13:09 schreef Joao Eduardo Luis <joao@xxxxxxx>: >>>>> >>>>> >>>>>> On 11/03/2016 09:40 AM, Wido den Hollander wrote: >>>>>> root@mon3:/var/lib/ceph/mon# ceph-monstore-tool ceph-mon3 dump-keys|awk '{print $1}'|uniq -c >>>>>> 96 auth >>>>>> 1143 logm >>>>>> 3 mdsmap >>>>>> 1 mkfs >>>>>> 1 mon_sync >>>>>> 6 monitor >>>>>> 3 monmap >>>>>> 1158 osdmap >>>>>> 358364 paxos >>>>>> 656 pgmap >>>>>> 6 pgmap_meta >>>>>> 168 pgmap_osd >>>>>> 6144 pgmap_pg >>>>>> root@mon3:/var/lib/ceph/mon# >>>>>> >>>>>> So there are 358k Paxos entries in the Mon store. >>>>>> >>>>>> Any suggestions on how to trim those from the MON store(s)? >>>>> >>>>> Can you check the value of paxos:first_committed in the store? >>>>> >>>> >>>> Here you go: >>>> >>>> root@mon3:~# ceph-monstore-tool /var/lib/ceph/mon/ceph-mon3 show-versions --map-type paxos >>>> first committed: 174349108 >>>> last committed: 174349609 >>>> root@mon3:~# >>>> >>>> Doesn't seem like a lot of keys in there? >>> >>> I have this annoying feeling that this relates to some really old bug I can't really recall the ticket number, but in essence was not trimming the maps even though first_committed was updated. >>> >>> The one way out of this I can think of is to change the value of 'first_committed' in the store to the very first paxos epoch you have. You will have to use 'ceph_kvstore_tool' to do that. >>> >> >> Is one mon enough? So stop that mon, change the value and start it again? >> >> Or do I need to do this on a mon which will become primary? > > Sorry for not being clear. This needs to be done on the one that will become the leader. > Ok! Np > Personally, I don't like this solution one bit, but I can't see any other way without a patched monitor, or maybe ceph_monstore_tool. > > If you are willing to wait till tomorrow, I'll be happy to kludge a sanitation feature onto ceph_monstore_tool that will clean those versions for you (latency being due to coding + testing + building). > Yes, no rush. I will wait. Thanks! Wido > -Joao _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com