Re: Monitors stores not trimming after upgrade from Dumpling to Hammer

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




> Op 3 nov. 2016 om 19:13 heeft Joao Eduardo Luis <joao@xxxxxxx> het volgende geschreven:
> 
>> On 11/03/2016 05:52 PM, wido@xxxxxxxx wrote:
>> 
>> 
>>>> Op 3 nov. 2016 om 16:44 heeft Joao Eduardo Luis <joao@xxxxxxx> het volgende geschreven:
>>>> 
>>>>> On 11/03/2016 01:24 PM, Wido den Hollander wrote:
>>>>> 
>>>>> Op 3 november 2016 om 13:09 schreef Joao Eduardo Luis <joao@xxxxxxx>:
>>>>> 
>>>>> 
>>>>>> On 11/03/2016 09:40 AM, Wido den Hollander wrote:
>>>>>> root@mon3:/var/lib/ceph/mon# ceph-monstore-tool ceph-mon3 dump-keys|awk '{print $1}'|uniq -c
>>>>>>    96 auth
>>>>>>  1143 logm
>>>>>>     3 mdsmap
>>>>>>     1 mkfs
>>>>>>     1 mon_sync
>>>>>>     6 monitor
>>>>>>     3 monmap
>>>>>>  1158 osdmap
>>>>>> 358364 paxos
>>>>>>   656 pgmap
>>>>>>     6 pgmap_meta
>>>>>>   168 pgmap_osd
>>>>>>  6144 pgmap_pg
>>>>>> root@mon3:/var/lib/ceph/mon#
>>>>>> 
>>>>>> So there are 358k Paxos entries in the Mon store.
>>>>>> 
>>>>>> Any suggestions on how to trim those from the MON store(s)?
>>>>> 
>>>>> Can you check the value of paxos:first_committed in the store?
>>>>> 
>>>> 
>>>> Here you go:
>>>> 
>>>> root@mon3:~# ceph-monstore-tool /var/lib/ceph/mon/ceph-mon3 show-versions --map-type paxos
>>>> first committed:    174349108
>>>> last  committed:    174349609
>>>> root@mon3:~#
>>>> 
>>>> Doesn't seem like a lot of keys in there?
>>> 
>>> I have this annoying feeling that this relates to some really old bug I can't really recall the ticket number, but in essence was not trimming the maps even though first_committed was updated.
>>> 
>>> The one way out of this I can think of is to change the value of 'first_committed' in the store to the very first paxos epoch you have. You will have to use 'ceph_kvstore_tool' to do that.
>>> 
>> 
>> Is one mon enough? So stop that mon, change the value and start it again?
>> 
>> Or do I need to do this on a mon which will become primary?
> 
> Sorry for not being clear. This needs to be done on the one that will become the leader.
> 

Ok! Np

> Personally, I don't like this solution one bit, but I can't see any other way without a patched monitor, or maybe ceph_monstore_tool.
> 
> If you are willing to wait till tomorrow, I'll be happy to kludge a sanitation feature onto ceph_monstore_tool that will clean those versions for you (latency being due to coding + testing + building).
> 

Yes, no rush. I will wait. Thanks!

Wido

>  -Joao
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux