Quoting Max Vernimmen (vernimmen@xxxxxxxxxxxxx): > > This is happening several times per day after we made several changes at > the same time: > > - add physical ram to the ceph nodes > - move from fixed 'bluestore cache size hdd|sdd' and 'bluestore cache kv > max' to 'bluestore cache autotune = 1' and 'osd memory target = > 20401094656'. > - update ceph from 12.2.8 to 12.2.11 > - update clients from 12.2.8 to 12.2.11 > > We have since upgraded the ceph nodes to 12.2.12 but it did not help to fix > this problem. Have you tried the new bitmap allocator for the OSDs already (available since 12.2.12): [osd] # MEMORY ALLOCATOR bluestore_allocator = bitmap bluefs_allocator = bitmap The issues you are reporting sound like an issue many of us have seen on luminous and mimic clusters and has been identified to be caused by the "stupid allocator" memory allocator. Gr. Stefan -- | BIT BV http://www.bit.nl/ Kamer van Koophandel 09090351 | GPG: 0xD14839C6 +31 318 648 688 / info@xxxxxx _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |