Hi,
We still suffer from a growing and growing monstore. Now we are at 80gb
and it just does not stop.
Any pointer would be appreciated. I opened bug:
https://tracker.ceph.com/issues/53485
Since we did not suffer this problem in the past, was some logging added
to mons in the recent releases ?
kind regards
poelzi
On 30/11/2021 02:59, Daniel Poelzleithner wrote:
Hi,
for reasons we are not sure yet, ours ceph-mon grew to wopping 40GB for
a small 6 node cluster. Rebuilding a mon causes the same size on the new
node. After dumping the monstore-tools:
# cat /tmp/ceph-mon-keys.log| awk '{print $1}' | uniq -c
200 auth
2 config
11 health
2036633 logm
146 mds_health
1 mds_metadata
682 mdsmap
948 mgr
1 mgr_command_descs
4 mgr_metadata
622928 mgrstat
2 mkfs
59 mon_config_key
1 mon_sync
7 monitor
1 monitor_store
53 monmap
21 osd_metadata
1 osd_pg_creating
4211 osd_snap
1209 osdmap
717 paxos
Is there a mechanism to burge those gazillon logm entries in a safe way ?
We even ran into storage problems because the mon was blowing up faster
then we could make space. Even after nodes run into health warings about
missing storage space on mon servers, some subsystem still decided to
spam the system into deadlock.
At least the growth stopped.
kind regards
poelzi
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
