Hi all! Here are again the good news and the bad news: BAD: another kernel memory leak detected (one more to hunt down and fix) GOOD: another kernel memory leak detected (one less unaccounted for) I tried to make some fun, but maintainers are busy folks, so let's get down to business: --- Nine (9) new systemd-udevd kernel memory leaks occurred (unable to reproduce). The platform is Ubuntu 22.10 with (relatively recent) systemd 251.4-1ubuntu7.1 on LENOVO_MT_82H8_BU_idea_FM_IdeaPad 3 15ITL6 with BIOS GGCN51WW from 11/16/2022. The symptom (/sys/kernel/debug/kmemleak output): unreferenced object 0xffff909698ff9280 (size 64): comm "systemd-udevd", pid 436, jiffies 4294893239 (age 6287.088s) hex dump (first 32 bytes): e0 51 bb 99 96 90 ff ff 00 00 00 00 00 00 00 00 .Q.............. 40 5b bb 99 96 90 ff ff 00 00 00 00 00 00 00 00 @[.............. backtrace: [<ffffffffb29de94c>] slab_post_alloc_hook+0x8c/0x320 [<ffffffffb29e5107>] __kmem_cache_alloc_node+0x1c7/0x2b0 [<ffffffffb2962f3b>] kmalloc_node_trace+0x2b/0xa0 [<ffffffffb31af2ec>] xhci_alloc_command+0x7c/0x1b0 [<ffffffffb31af451>] xhci_alloc_command_with_ctx+0x21/0x70 [<ffffffffb31a8a3e>] xhci_change_max_exit_latency+0x2e/0x1c0 [<ffffffffb31a8c5b>] xhci_disable_usb3_lpm_timeout+0x7b/0xb0 [<ffffffffb31457a7>] usb_disable_link_state+0x57/0xe0 [<ffffffffb3145f46>] usb_disable_lpm+0x86/0xc0 [<ffffffffb3145fc1>] usb_unlocked_disable_lpm+0x31/0x60 [<ffffffffb3155db6>] usb_disable_device+0x136/0x250 [<ffffffffb3156b23>] usb_set_configuration+0x583/0xa70 [<ffffffffb3164c6d>] usb_generic_driver_disconnect+0x2d/0x40 [<ffffffffb3158612>] usb_unbind_device+0x32/0x90 [<ffffffffb3022295>] device_remove+0x65/0x70 [<ffffffffb3023903>] device_release_driver_internal+0xc3/0x140 unreferenced object 0xffff909699bb5b40 (size 32): comm "systemd-udevd", pid 436, jiffies 4294893239 (age 6287.088s) hex dump (first 32 bytes): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 50 5b bb 99 96 90 ff ff 50 5b bb 99 96 90 ff ff P[......P[...... backtrace: [<ffffffffb29de94c>] slab_post_alloc_hook+0x8c/0x320 [<ffffffffb29e5107>] __kmem_cache_alloc_node+0x1c7/0x2b0 [<ffffffffb2962f3b>] kmalloc_node_trace+0x2b/0xa0 [<ffffffffb31af364>] xhci_alloc_command+0xf4/0x1b0 [<ffffffffb31af451>] xhci_alloc_command_with_ctx+0x21/0x70 [<ffffffffb31a8a3e>] xhci_change_max_exit_latency+0x2e/0x1c0 [<ffffffffb31a8c5b>] xhci_disable_usb3_lpm_timeout+0x7b/0xb0 [<ffffffffb31457a7>] usb_disable_link_state+0x57/0xe0 [<ffffffffb3145f46>] usb_disable_lpm+0x86/0xc0 [<ffffffffb3145fc1>] usb_unlocked_disable_lpm+0x31/0x60 [<ffffffffb3155db6>] usb_disable_device+0x136/0x250 [<ffffffffb3156b23>] usb_set_configuration+0x583/0xa70 [<ffffffffb3164c6d>] usb_generic_driver_disconnect+0x2d/0x40 [<ffffffffb3158612>] usb_unbind_device+0x32/0x90 [<ffffffffb3022295>] device_remove+0x65/0x70 [<ffffffffb3023903>] device_release_driver_internal+0xc3/0x140 unreferenced object 0xffff909699bb51e0 (size 32): comm "systemd-udevd", pid 436, jiffies 4294893239 (age 6287.088s) hex dump (first 32 bytes): 02 00 00 00 20 04 00 00 00 a0 ff 98 96 90 ff ff .... ........... 00 a0 ff 18 01 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<ffffffffb29de94c>] slab_post_alloc_hook+0x8c/0x320 [<ffffffffb29e5107>] __kmem_cache_alloc_node+0x1c7/0x2b0 [<ffffffffb2962f3b>] kmalloc_node_trace+0x2b/0xa0 [<ffffffffb31ad86e>] xhci_alloc_container_ctx+0x7e/0x140 [<ffffffffb31af469>] xhci_alloc_command_with_ctx+0x39/0x70 [<ffffffffb31a8a3e>] xhci_change_max_exit_latency+0x2e/0x1c0 [<ffffffffb31a8c5b>] xhci_disable_usb3_lpm_timeout+0x7b/0xb0 [<ffffffffb31457a7>] usb_disable_link_state+0x57/0xe0 [<ffffffffb3145f46>] usb_disable_lpm+0x86/0xc0 [<ffffffffb3145fc1>] usb_unlocked_disable_lpm+0x31/0x60 [<ffffffffb3155db6>] usb_disable_device+0x136/0x250 [<ffffffffb3156b23>] usb_set_configuration+0x583/0xa70 [<ffffffffb3164c6d>] usb_generic_driver_disconnect+0x2d/0x40 [<ffffffffb3158612>] usb_unbind_device+0x32/0x90 [<ffffffffb3022295>] device_remove+0x65/0x70 [<ffffffffb3023903>] device_release_driver_internal+0xc3/0x140 . . . Please find the config, lshw output and complete /sys/kernel/debug/kmemleak output here: https://domac.alu.unizg.hr/~mtodorov/linux/bugreports/systemd-udevd/kmemleak.log https://domac.alu.unizg.hr/~mtodorov/linux/bugreports/systemd-udevd/lshw.txt https://domac.alu.unizg.hr/~mtodorov/linux/bugreports/systemd-udevd/config-6.3.0-rc3-kobj-rlse-00317-g65aca32efdcb The systemd issue tracker said they accept issues only for the most recent 253 and 252, 251.4 seems too old for them despite being issued on May 21, 2022 (Source: https://github.com/systemd/systemd/releases). It is not that I want to dump this on Linux kernel developers, but I felt like it is a kernel memory leak problem rather than a bug in systemd-udevd. Of course, my hunch might be wrong ... As per Code of Conduct, I have checked for the developers and maintainers with scripts/get_maintainers.pl. Best regards, Mirsad -- Mirsad Goran Todorovac Sistem inženjer Grafički fakultet | Akademija likovnih umjetnosti Sveučilište u Zagrebu System engineer Faculty of Graphic Arts | Academy of Fine Arts University of Zagreb, Republic of Croatia The European Union