Moin A customer reported panics inside memory management. Before all occurances there are reports about SLAB missmatch in the log. The "crash" tool shows freelist corruption in the memory dump. This makes this problem a use-after-free somewhere inside the ceph module. The crashs happen during high load situations, while copying data between two cephfs. | [152791.777454] ceph: dropping dirty+flushing - state for 00000000c039d4cc 1099526092092 | [152791.777457] ------------[ cut here ]------------ | [152791.777458] cache_from_obj: Wrong slab cache. jbd2_journal_handle but object is from kmalloc-256 | [152791.777473] WARNING: CPU: 76 PID: 2676615 at mm/slab.h:521 kmem_cache_free+0x260/0x2b0 […] | [152791.777530] CPU: 76 PID: 2676615 Comm: kworker/76:2 Kdump: loaded Not tainted 5.4.0-81-generic #91-Ubuntu | [152791.777531] Hardware name: HPE ProLiant DL385 Gen10/ProLiant DL385 Gen10, BIOS A40 10/28/2021 | [152791.777540] Workqueue: ceph-msgr ceph_con_workfn [libceph] | [152791.777542] RIP: 0010:kmem_cache_free+0x260/0x2b0 […] | [152791.777550] Call Trace: | [152791.777562] ceph_free_cap_flush+0x1d/0x20 [ceph] | [152791.777568] remove_session_caps_cb+0xcf/0x4b0 [ceph] | [152791.777573] ceph_iterate_session_caps+0xc8/0x2a0 [ceph] | [152791.777578] ? wake_up_session_cb+0xe0/0xe0 [ceph] | [152791.777583] remove_session_caps+0x55/0x190 [ceph] | [152791.777587] ? cleanup_session_requests+0x104/0x130 [ceph] | [152791.777592] handle_session+0x4c7/0x5e0 [ceph] | [152791.777597] dispatch+0x279/0x610 [ceph] | [152791.777602] try_read+0x566/0x8c0 [libceph] They reported the same in all tested kernels since 5.4, up to 5.15.5 or so. Sadly I have no tests with newer builds available. Any ideas how I can debug this further? Regards, Bastian -- Bastian Blank Berater Telefon: +49 2166 9901-194 E-Mail: bastian.blank@xxxxxxxxxxx credativ GmbH, HRB Mönchengladbach 12080, USt-ID-Nummer: DE204566209 Trompeterallee 108, 41189 Mönchengladbach Geschäftsführung: Dr. Michael Meskes, Geoff Richardson, Peter Lilley Unser Umgang mit personenbezogenen Daten unterliegt folgenden Bestimmungen: https://www.credativ.de/datenschutz
