Hi, sorry for my late reply, Enabling CONFIG_DEBUG_ATOMIC_SLEEP and CONFIG_PROVE_LOCKING the kernel image became too big and overlapped the DT once loaded. So I went through menuconfig and disabled many unused drivers and features. Then for some days, I couldn't get any kernel panics at all (I think I hadn't left the system running long enough). In the past week, I've collected the three attached panic logs. I can't see any correlation among the logs. I'd say there are some hardware issues, but it couldn't be the case since I know the same board works fine with a non-RT image.
Il 18/06/24 16:19, Kurt Kanzenbach ha
scritto:
Hi Riccardo, On Tue Jun 18 2024, Riccardo Laiolo wrote:Hi all, I'm working on a custom board with an NXP iMX8MP cpu (linux 6.1.55-rt16 from NXP's yocto repository). The hardware is similar to the NXP evk. The main difference is the dsa switch (marvell MV88E6390). If I enable the marvell's DT node the system becomes unstable. I have random kernel panics.Hm. I've never observed any issues with this switch driver or DSA and PREEMPT_RT in general.The error trace is not helpful, it looks like a stack overflow or a stack corruption. Sometimes it's a NULL (or random unallocated address) pointer dereference, but sometimes the error is different, I've seen an error about killing the idle task a couple of times. What seems constant is that the error occurs during the scheduler context switch after an hard interrupt. How can I troubleshoot this kind of issues?Can you enable CONFIG_DEBUG_ATOMIC_SLEEP, CONFIG_PROVE_LOCKING and see if anything unusual comes up? Thanks, Kurt
[ 2857.996307] ------------[ cut here ]------------ [ 2857.996316] Current state: 0 [ 2857.996336] WARNING: CPU: 0 PID: 0 at kernel/time/clockevents.c:319 clockevents_program_event+0x124/0x130 [ 2857.996352] Modules linked in: fsl_jr_uio caam_jr caamkeyblob_desc caamhash_desc caamalg_desc crypto_engine authenc libdes snd_soc_fsl_sai crct10dif_ce snd_soc_fsl_utils polyval_ce cfg80211 polyval_generic caam secvio error ebtables x_tables overlay g_ether leaff_rpmsg fuse [ 2857.996394] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 6.1.55-rt16 #1 [ 2857.996399] Hardware name: LEAFF iMX8 M7 controller board (DT) [ 2857.996402] pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 2857.996407] pc : clockevents_program_event+0x124/0x130 [ 2857.996411] lr : clockevents_program_event+0x124/0x130 [ 2857.996414] sp : ffff800009d23c40 [ 2857.996416] x29: ffff800009d23c40 x28: 0000000000000001 x27: 0000000000000001 [ 2857.996422] x26: ffff0001f6f8d040 x25: ffff800009a28040 x24: 000000000000000a [ 2857.996428] x23: 00000299712c9900 x22: ffff800009dd7740 x21: 0000000000000001 [ 2857.996436] x20: 00000299712c9900 x19: ffff0001f6fa0dc0 x18: ffffffffffffffff [ 2857.996444] x17: 000000040044ffff x16: 0000000000000954 x15: ffff800009d237f0 [ 2857.996449] x14: 0000000000000000 x13: ffff80000aa5d78f x12: 2d2d2d2d5d206572 [ 2857.996457] x11: fffffffffffe0000 x10: 000000000000000a x9 : ffff800009d23c40 [ 2857.996468] x8 : 000000000000000a x7 : 0000000000000001 x6 : 000000000000000a [ 2857.996473] x5 : ffff0001f6f89ec8 x4 : 0000000000000000 x3 : 0000000000000027 [ 2857.996479] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff800009d41500 [ 2857.996488] Call trace: [ 2857.996489] clockevents_program_event+0x124/0x130 [ 2857.996493] tick_program_event+0x58/0xa4 [ 2857.996499] hrtimer_start_range_ns+0x304/0x34c [ 2857.996506] tick_nohz_stop_tick+0x108/0x1d0 [ 2857.996511] tick_nohz_idle_stop_tick+0x78/0xd4 [ 2857.996516] do_idle+0x244/0x310 [ 2857.996524] cpu_startup_entry+0x24/0x30 [ 2857.996527] rest_init+0x108/0x1a0 [ 2857.996534] arch_post_acpi_subsys_init+0x0/0x18 [ 2857.996542] start_kernel+0x70c/0x750 [ 2857.996547] __primary_switched+0xbc/0xc4 [ 2857.996554] irq event stamp: 2694162 [ 2857.996556] hardirqs last enabled at (2694161): [<ffff800008138c34>] tick_nohz_idle_enter+0x70/0x16c [ 2857.996563] hardirqs last disabled at (2694162): [<ffff8000080afe44>] do_idle+0xf4/0x310 [ 2857.996570] softirqs last enabled at (3296): [<ffff80000805cf08>] __local_bh_enable_ip+0x98/0x240 [ 2857.996579] softirqs last disabled at (3286): [<ffff80000814ac20>] cgroup_idr_alloc.constprop.0+0x0/0xc0 [ 2857.996586] ---[ end trace 0000000000000000 ]--- [ 2857.996592] Unable to handle kernel execute from non-executable memory at virtual address ffff80000aa4bc20 [ 2857.996595] Mem abort info: [ 2857.996596] ESR = 0x000000008600000f [ 2857.996598] EC = 0x21: IABT (current EL), IL = 32 bits [ 2857.996600] SET = 0, FnV = 0 [ 2857.996602] EA = 0, S1PTW = 0 [ 2857.996603] FSC = 0x0f: level 3 permission fault [ 2857.996606] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000041d1c000 [ 2857.996609] [ffff80000aa4bc20] pgd=100000023ffff003, p4d=100000023ffff003, pud=100000023fffe003, pmd=100000023fff9003, pte=0078000042e4b703 [ 2857.996620] Internal error: Oops: 000000008600000f [#1] PREEMPT_RT SMP [ 2857.996624] Modules linked in: fsl_jr_uio caam_jr caamkeyblob_desc caamhash_desc caamalg_desc crypto_engine authenc libdes snd_soc_fsl_sai crct10dif_ce snd_soc_fsl_utils polyval_ce cfg80211 polyval_generic caam secvio error ebtables x_tables overlay g_ether leaff_rpmsg fuse [ 2857.996660] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 6.1.55-rt16 #1 [ 2857.996665] Hardware name: LEAFF iMX8 M7 controller board (DT) [ 2857.996666] pstate: a00000c5 (NzCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 2857.996671] pc : __key.3+0x0/0x10 [ 2857.996678] lr : clockevents_program_event+0xa8/0x130 [ 2857.996682] sp : ffff800009d23c40 [ 2857.996683] x29: ffff800009d23c40 x28: 0000000000000001 x27: 0000000000000001 [ 2857.996689] x26: ffff0001f6f8d040 x25: ffff800009a28040 x24: 000000000000000a [ 2857.996694] x23: 00000299712c9900 x22: ffff800009dd7740 x21: 0000000000000001 [ 2857.996700] x20: 000000000070239b x19: ffff0001f6fa0dc0 x18: ffffffffffffffff [ 2857.996708] x17: 000000040044ffff x16: 0000000000000954 x15: ffff800009d237f0 [ 2857.996716] x14: 0000000000000000 x13: ffff80000aa5d78f x12: 2d2d2d2d5d206572 [ 2857.996722] x11: fffffffffffe0000 x10: 000000000000000a x9 : ffff800009d23c40 [ 2857.996728] x8 : ffff800009a23008 x7 : 0000000000000001 x6 : 000000055e6fef2a [ 2857.996733] x5 : 00ffffffffffffff x4 : 0000000000000000 x3 : 0000000000000000 [ 2857.996741] x2 : ffff80000aa4bc20 x1 : ffff0001f6fa0dc0 x0 : 0000000000000000 [ 2857.996747] Call trace: [ 2857.996748] __key.3+0x0/0x10 [ 2857.996753] tick_program_event+0x58/0xa4 [ 2857.996757] hrtimer_start_range_ns+0x304/0x34c [ 2857.996763] tick_nohz_stop_tick+0x108/0x1d0 [ 2857.996768] tick_nohz_idle_stop_tick+0x78/0xd4 [ 2857.996773] do_idle+0x244/0x310 [ 2857.996778] cpu_startup_entry+0x24/0x30 [ 2857.996784] rest_init+0x108/0x1a0 [ 2857.996789] arch_post_acpi_subsys_init+0x0/0x18 [ 2857.996795] start_kernel+0x70c/0x750 [ 2857.996800] __primary_switched+0xbc/0xc4 [ 2857.996807] Code: 00000000 00000000 00000000 00000000 (00000000) [ 2858.472437] ---[ end trace 0000000000000000 ]--- [ 2858.472442] Kernel panic - not syncing: Attempted to kill the idle task! [ 2858.472446] SMP: stopping secondary CPUs [ 2858.472749] Kernel Offset: disabled [ 2858.472750] CPU features: 0x00000,00800084,0000420b [ 2858.472754] Memory Limit: none [ 2858.499390] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]---
[ 4386.503700] Unable to handle kernel paging request at virtual address 0000000000003fb8
[ 4386.503714] Mem abort info:
[ 4386.503715] ESR = 0x0000000096000004
[ 4386.503717] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503721] SET = 0, FnV = 0
[ 4386.503722] EA = 0, S1PTW = 0
[ 4386.503724] FSC = 0x04: level 0 translation fault
[ 4386.503726] Data abort info:
[ 4386.503727] ISV = 0, ISS = 0x00000004
[ 4386.503728] CM = 0, WnR = 0
[ 4386.503733] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503736] Mem abort info:
[ 4386.503737] ESR = 0x0000000096000004
[ 4386.503738] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503740] SET = 0, FnV = 0
[ 4386.503742] EA = 0, S1PTW = 0
[ 4386.503743] FSC = 0x04: level 0 translation fault
[ 4386.503745] Data abort info:
[ 4386.503746] ISV = 0, ISS = 0x00000004
[ 4386.503748] CM = 0, WnR = 0
[ 4386.503751] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503753] Mem abort info:
[ 4386.503754] ESR = 0x0000000096000004
[ 4386.503755] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503757] SET = 0, FnV = 0
[ 4386.503759] EA = 0, S1PTW = 0
[ 4386.503761] FSC = 0x04: level 0 translation fault
[ 4386.503762] Data abort info:
[ 4386.503763] ISV = 0, ISS = 0x00000004
[ 4386.503765] CM = 0, WnR = 0
[ 4386.503768] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503770] Mem abort info:
[ 4386.503771] ESR = 0x0000000096000004
[ 4386.503772] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503775] SET = 0, FnV = 0
[ 4386.503776] EA = 0, S1PTW = 0
[ 4386.503778] FSC = 0x04: level 0 translation fault
[ 4386.503780] Data abort info:
[ 4386.503780] ISV = 0, ISS = 0x00000004
[ 4386.503782] CM = 0, WnR = 0
[ 4386.503785] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503787] Mem abort info:
[ 4386.503788] ESR = 0x0000000096000004
[ 4386.503789] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503792] SET = 0, FnV = 0
[ 4386.503793] EA = 0, S1PTW = 0
[ 4386.503795] FSC = 0x04: level 0 translation fault
[ 4386.503797] Data abort info:
[ 4386.503798] ISV = 0, ISS = 0x00000004
[ 4386.503799] CM = 0, WnR = 0
[ 4386.503802] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503804] Mem abort info:
[ 4386.503805] ESR = 0x0000000096000004
[ 4386.503807] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503809] SET = 0, FnV = 0
[ 4386.503811] EA = 0, S1PTW = 0
[ 4386.503812] FSC = 0x04: level 0 translation fault
[ 4386.503814] Data abort info:
[ 4386.503815] ISV = 0, ISS = 0x00000004
[ 4386.503817] CM = 0, WnR = 0
[ 4386.503819] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503822] Mem abort info:
[ 4386.503823] ESR = 0x0000000096000004
[ 4386.503824] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503826] SET = 0, FnV = 0
[ 4386.503828] EA = 0, S1PTW = 0
[ 4386.503830] FSC = 0x04: level 0 translation fault
[ 4386.503831] Data abort info:
[ 4386.503832] ISV = 0, ISS = 0x00000004
[ 4386.503834] CM = 0, WnR = 0
[ 4386.503836] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503839] Mem abort info:
[ 4386.503840] ESR = 0x0000000096000004
[ 4386.503841] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503843] SET = 0, FnV = 0
[ 4386.503845] EA = 0, S1PTW = 0
[ 4386.503847] FSC = 0x04: level 0 translation fault
[ 4386.503848] Data abort info:
[ 4386.503849] ISV = 0, ISS = 0x00000004
[ 4386.503851] CM = 0, WnR = 0
[ 4386.503853] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503855] Mem abort info:
[ 4386.503857] ESR = 0x0000000096000004
[ 4386.503858] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503860] SET = 0, FnV = 0
[ 4386.503862] EA = 0, S1PTW = 0
[ 4386.503863] FSC = 0x04: level 0 translation fault
[ 4386.503865] Data abort info:
[ 4386.503866] ISV = 0, ISS = 0x00000004
[ 4386.503867] CM = 0, WnR = 0
[ 4386.503871] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503873] Mem abort info:
[ 4386.503874] ESR = 0x0000000096000004
[ 4386.503875] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503878] SET = 0, FnV = 0
[ 4386.503879] EA = 0, S1PTW = 0
[ 4386.503881] FSC = 0x04: level 0 translation fault
[ 4386.503883] Data abort info:
[ 4386.503884] ISV = 0, ISS = 0x00000004
[ 4386.503885] CM = 0, WnR = 0
[ 4386.503888] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503890] Mem abort info:
[ 4386.503891] ESR = 0x0000000096000004
[ 4386.503893] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503895] SET = 0, FnV = 0
[ 4386.503896] EA = 0, S1PTW = 0
[ 4386.503898] FSC = 0x04: level 0 translation fault
[ 4386.503900] Data abort info:
[ 4386.503901] ISV = 0, ISS = 0x00000004
[ 4386.503902] CM = 0, WnR = 0
[ 4386.503905] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503907] Mem abort info:
[ 4386.503908] ESR = 0x0000000096000004
[ 4386.503909] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503912] SET = 0, FnV = 0
[ 4386.503913] EA = 0, S1PTW = 0
[ 4386.503915] FSC = 0x04: level 0 translation fault
[ 4386.503917] Data abort info:
[ 4386.503918] ISV = 0, ISS = 0x00000004
[ 4386.503919] CM = 0, WnR = 0
[ 4386.503922] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503924] Mem abort info:
[ 4386.503925] ESR = 0x0000000096000004
[ 4386.503927] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503929] SET = 0, FnV = 0
[ 4386.503931] EA = 0, S1PTW = 0
[ 4386.503932] FSC = 0x04: level 0 translation fault
[ 4386.503934] Data abort info:
[ 4386.503935] ISV = 0, ISS = 0x00000004
[ 4386.503936] CM = 0, WnR = 0
[ 4386.503939] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503941] Mem abort info:
[ 4386.503942] ESR = 0x0000000096000004
[ 4386.503944] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503946] SET = 0, FnV = 0
[ 4386.503947] EA = 0, S1PTW = 0
[ 4386.503949] FSC = 0x04: level 0 translation fault
[ 4386.503951] Data abort info:
[ 4386.503952] ISV = 0, ISS = 0x00000004
[ 4386.503953] CM = 0, WnR = 0
[ 4386.503956] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503958] Mem abort info:
[ 4386.503959] ESR = 0x0000000096000004
[ 4386.503961] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503963] SET = 0, FnV = 0
[ 4386.503965] EA = 0, S1PTW = 0
[ 4386.503966] FSC = 0x04: level 0 translation fault
[ 4386.503968] Data abort info:
[ 4386.503969] ISV = 0, ISS = 0x00000004
[ 4386.503970] CM = 0, WnR = 0
[ 4386.503973] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503975] Mem abort info:
[ 4386.503976] ESR = 0x0000000096000004
[ 4386.503978] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503980] SET = 0, FnV = 0
[ 4386.503982] EA = 0, S1PTW = 0
[ 4386.503983] FSC = 0x04: level 0 translation fault
[ 4386.503985] Data abort info:
[ 4386.503986] ISV = 0, ISS = 0x00000004
[ 4386.503987] CM = 0, WnR = 0
[ 4386.503990] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.503992] Mem abort info:
[ 4386.503994] ESR = 0x0000000096000004
[ 4386.503995] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.503997] SET = 0, FnV = 0
[ 4386.503999] EA = 0, S1PTW = 0
[ 4386.504001] FSC = 0x04: level 0 translation fault
[ 4386.504002] Data abort info:
[ 4386.504003] ISV = 0, ISS = 0x00000004
[ 4386.504004] CM = 0, WnR = 0
[ 4386.504007] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.504010] Mem abort info:
[ 4386.504011] ESR = 0x0000000096000004
[ 4386.504012] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.504014] SET = 0, FnV = 0
[ 4386.504016] EA = 0, S1PTW = 0
[ 4386.504017] FSC = 0x04: level 0 translation fault
[ 4386.504019] Data abort info:
[ 4386.504020] ISV = 0, ISS = 0x00000004
[ 4386.504022] CM = 0, WnR = 0
[ 4386.504025] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.504027] Mem abort info:
[ 4386.504028] ESR = 0x0000000096000004
[ 4386.504029] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.504031] SET = 0, FnV = 0
[ 4386.504033] EA = 0, S1PTW = 0
[ 4386.504035] FSC = 0x04: level 0 translation fault
[ 4386.504036] Data abort info:
[ 4386.504037] ISV = 0, ISS = 0x00000004
[ 4386.504039] CM = 0, WnR = 0
[ 4386.504041] Unable to handle kernel NULL pointer dereference at virtual address 00000000000000b0
[ 4386.504044] Mem abort info:
[ 4386.504044] ESR = 0x0000000096000004
[ 4386.504046] EC = 0x25: DABT (current EL), IL = 32 bits
[ 4386.504048] SET = 0, FnV = 0
[ 4386.504050] EA = 0, S1PTW = 0
[ 4386.504051] FSC = 0x04: level 0 translation fault
[ 4386.504053] Data abort info:
[ 4386.504054] ISV = 0, ISS = 0x00000004
[ 4386.504055] CM = 0, WnR = 0
[ 4386.504061] Insufficient stack space to handle exception!
[ 4386.504062] ESR: 0x0000000096000047 -- DABT (current EL)
[ 4386.504065] FAR: 0xffff80000acfffe0
[ 4386.504066] Task stack: [0xffff80000ad00000..0xffff80000ad04000]
[ 4386.504068] IRQ stack: [0xffff800008008000..0xffff80000800c000]
[ 4386.504071] Overflow stack: [0xffff0001f6fa6300..0xffff0001f6fa7300]
[ 4386.504075] CPU: 1 PID: 26 Comm: ksoftirqd/1 Tainted: G W 6.1.55-rt16 #1
[ 4386.504080] Hardware name: LEAFF iMX8 M7 controller board (DT)
[ 4386.504083] pstate: a00000c5 (NzCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 4386.504088] pc : number+0x18/0x3bc
[ 4386.504100] lr : vsnprintf+0x2f8/0x6f0
[ 4386.504105] sp : ffff80000acfffc0
[ 4386.504107] x29: ffff80000ad00050 x28: ffff8000096f7666 x27: 00000000ffffffd8
[ 4386.504115] x26: ffff80000ad00360 x25: 0000000000000008 x24: ffff80000ad00360
[ 4386.504121] x23: ffff80000ad00160 x22: ffff80000ad00168 x21: 0000000000000006
[ 4386.504129] x20: ffff80000ad001a6 x19: ffff8000096f7666 x18: ffffffffffffffff
[ 4386.504137] x17: 72646461206c6175 x16: 7472697620746120 x15: ffff80000acfffc0
[ 4386.504143] x14: ffff80000ad00168 x13: ffff80000ad001a6 x12: 2c30203d204d4320
[ 4386.504150] x11: fffffffffffe0000 x10: 0000000000000000 x9 : ffff80000ad00360
[ 4386.504155] x8 : ffff80000ad00360 x7 : 00000000ffffffff x6 : ffff80000ad00168
[ 4386.504161] x5 : 0000000000000018 x4 : ffff0000d0254bc0 x3 : ffff103000001009
[ 4386.504167] x2 : 00000000000000b0 x1 : ffff80000ad00168 x0 : ffff80000ad001a6
[ 4386.504174] Kernel panic - not syncing: kernel stack overflow
[ 4386.504178] SMP: stopping secondary CPUs
[ 4387.504180] SMP: failed to stop secondary CPUs 0-2
[ 4387.504188] Kernel Offset: disabled
[ 4387.504190] CPU features: 0x00000,00800084,0000420b
[ 4387.504192] Memory Limit: none
[ 4387.504197]
[ 4387.504198] ================================
[ 4387.504199] WARNING: inconsistent lock state
[ 4387.504201] 6.1.55-rt16 #1 Tainted: G W
[ 4387.504203] --------------------------------
[ 4387.504204] inconsistent {INITIAL USE} -> {IN-NMI} usage.
[ 4387.504207] ksoftirqd/1/26 [HC1[1]:SC0[0]:HE0:SE1] takes:
[ 4387.504211] ffff800009d58ca8 ((console_sem).lock){....}-{2:2}, at: down_trylock+0x18/0x50
[ 4387.504225] {INITIAL USE} state was registered at:
[ 4387.504227] lock_acquire+0x230/0x34c
[ 4387.504234] _raw_spin_lock_irqsave+0x70/0xb0
[ 4387.504241] down+0x2c/0x80
[ 4387.504244] console_lock+0x4c/0x13c
[ 4387.504250] con_init+0x28/0x254
[ 4387.504255] console_init+0x15c/0x208
[ 4387.504262] start_kernel+0x628/0x750
[ 4387.504268] __primary_switched+0xbc/0xc4
[ 4387.504273] irq event stamp: 2948138
[ 4387.504274] hardirqs last enabled at (2948137): [<ffff800008093bd4>] finish_task_switch.isra.0+0xb4/0x2d0
[ 4387.504283] hardirqs last disabled at (2948138): [<ffff800009211408>] __schedule+0x578/0x8d0
[ 4387.504288] softirqs last enabled at (2948132): [<ffff80000805d2a8>] run_ksoftirqd+0xb4/0x120
[ 4387.504296] softirqs last disabled at (2948126): [<ffff80000808d370>] smpboot_thread_fn+0x220/0x320
[ 4387.504301]
[ 4387.504301] other info that might help us debug this:
[ 4387.504302] Possible unsafe locking scenario:
[ 4387.504302]
[ 4387.504304] CPU0
[ 4387.504304] ----
[ 4387.504305] lock((console_sem).lock);
[ 4387.504308] <Interrupt>
[ 4387.504309] lock((console_sem).lock);
[ 4387.504312]
[ 4387.504312] *** DEADLOCK ***
[ 4387.504312]
[ 4387.504313] no locks held by ksoftirqd/1/26.
[ 4387.504315]
[ 4387.504315] stack backtrace:
[ 4387.504317] CPU: 1 PID: 26 Comm: ksoftirqd/1 Tainted: G W 6.1.55-rt16 #1
[ 4387.504321] Hardware name: LEAFF iMX8 M7 controller board (DT)
[ 4387.504323] Call trace:
[ 4387.504325] dump_backtrace.part.0+0xe4/0xf0
[ 4387.504332] show_stack+0x18/0x30
[ 4387.504338] dump_stack_lvl+0x88/0xb4
[ 4387.504344] dump_stack+0x18/0x34
[ 4387.504348] print_usage_bug.part.0+0x204/0x228
[ 4387.504353] lock_acquire+0x310/0x34c
[ 4387.504358] _raw_spin_lock_irqsave+0x70/0xb0
[ 4387.504363] down_trylock+0x18/0x50
[ 4387.504369] __down_trylock_console_sem+0x3c/0xc0
[ 4387.504373] console_unblank+0x3c/0x11c
[ 4387.504378] panic+0x234/0x38c
[ 4387.504382] nmi_panic+0xac/0xb0
[ 4387.504386] panic_bad_stack+0x118/0x13c
[ 4387.504390] handle_bad_stack+0x34/0x130
[ 4387.504395] __bad_stack+0x78/0x7c
[ 4387.504399] number+0x18/0x3bc
[ 4387.504403] vprintk_store+0x114/0x48c
[ 4387.504407] vprintk_emit+0x80/0x220
[ 4387.504411] vprintk_default+0x38/0x44
[ 4387.504418] vprintk+0x88/0xe0
[ 4387.504422] _printk+0x60/0x8c
[ 4387.504426] die_kernel_fault+0x48/0x364
[ 4387.504430] do_alignment_fault+0x0/0x80
[ 4387.504436] do_page_fault+0xb8/0x380
[ 4387.504441] do_translation_fault+0x9c/0xb0
[ 4387.504449] do_mem_abort+0x44/0x94
[ 4387.504454] el1_abort+0x44/0x70
[ 4387.504459] el1h_64_sync_handler+0xd8/0xe4
[ 4387.504464] el1h_64_sync+0x64/0x68
[ 4387.504467] die_kernel_fault+0x1cc/0x364
[ 4387.504471] do_alignment_fault+0x0/0x80
[ 4387.504476] do_page_fault+0xb8/0x380
[ 4387.504484] do_translation_fault+0x9c/0xb0
[ 4387.504489] do_mem_abort+0x44/0x94
[ 4387.504497] el1_abort+0x44/0x70
[ 4387.504501] el1h_64_sync_handler+0xd8/0xe4
[ 4387.504508] el1h_64_sync+0x64/0x68
[ 4387.504512] die_kernel_fault+0x1cc/0x364
[ 4387.504515] do_alignment_fault+0x0/0x80
[ 4387.504521] do_page_fault+0xb8/0x380
[ 4387.504528] do_translation_fault+0x9c/0xb0
[ 4387.504534] do_mem_abort+0x44/0x94
[ 4387.504539] el1_abort+0x44/0x70
[ 4387.504545] el1h_64_sync_handler+0xd8/0xe4
[ 4387.504550] el1h_64_sync+0x64/0x68
[ 4387.504556] die_kernel_fault+0x1cc/0x364
[ 4387.504560] do_alignment_fault+0x0/0x80
[ 4387.504565] do_page_fault+0xb8/0x380
[ 4387.504573] do_translation_fault+0x9c/0xb0
[ 4387.504578] do_mem_abort+0x44/0x94
[ 4387.504583] el1_abort+0x44/0x70
[ 4387.504588] el1h_64_sync_handler+0xd8/0xe4
[ 4387.504592] el1h_64_sync+0x64/0x68
[ 4387.504598] die_kernel_fault+0x1cc/0x364
[ 4387.504602] do_alignment_fault+0x0/0x80
[ 4387.504607] do_page_fault+0xb8/0x380
[ 4387.504612] do_translation_fault+0x9c/0xb0
[ 4387.504618] do_mem_abort+0x44/0x94
[ 4387.504623] el1_abort+0x44/0x70
[ 4387.504629] el1h_64_sync_handler+0xd8/0xe4
[ 4387.504636] el1h_64_sync+0x64/0x68
[ 4387.504640] die_kernel_fault+0x1cc/0x364
[ 4387.504643] do_alignment_fault+0x0/0x80
[ 4387.504649] do_page_fault+0xb8/0x380
[ 4387.504654] do_translation_fault+0x9c/0xb0
[ 4387.504660] do_mem_abort+0x44/0x94
[ 4387.504665] el1_abort+0x44/0x70
[ 4387.504672] el1h_64_sync_handler+0xd8/0xe4
[ 4387.504676] el1h_64_sync+0x64/0x68
[ 4387.504680] die_kernel_fault+0x1cc/0x364
[ 4387.504683] do_alignment_fault+0x0/0x80
[ 4387.504689] do_page_fault+0xb8/0x380
[ 4387.504694] do_translation_fault+0x9c/0xb0
[ 4387.504702] do_mem_abort+0x44/0x94
[ 4387.504707] el1_abort+0x44/0x70
[ 4387.504711] el1h_64_sync_handler+0xd8/0xe4
[ 4387.504716] el1h_64_sync+0x64/0x68
[ 4387.504719] die_kernel_fault+0x1cc/0x364
[ 4387.504723] do_alignment_fault+0x0/0x80
[ 4387.504731] do_page_fault+0xb8/0x380
[ 4387.504736] do_translation_fault+0x9c/0xb0
[ 4387.504741] do_mem_abort+0x44/0x94
[ 4387.504749] el1_abort+0x44/0x70
[ 4387.504753] el1h_64_sync_handler+0xd8/0xe4
[ 4387.504760] el1h_64_sync+0x64/0x68
[ 4387.504764] die_kernel_fault+0x1cc/0x364
[ 4387.504767] do_alignment_fault+0x0/0x80
[ 4387.504773] do_page_fault+0xb8/0x380
[ 4387.504778] do_translation_fault+0x9c/0xb0
[ 4387.504784] do_mem_abort+0x44/0x94
[ 4387.504789] el1_abort+0x44/0x70
[ 4387.504793] el1h_64_sync_handler+0xd8/0xe4
[ 4387.504798] el1h_64_sync+0x64/0x68
[ 4387.504803] die_kernel_fault+0x1cc/0x364
[ 4387.504807] do_alignment_fault+0x0/0x80
[ 4387.504815] do_page_fault+0xb8/0x380
[ 4387.504822] do_translation_fault+0x9c/0xb0
[ 4387.504828] do_mem_abort+0x44/0x94
[ 4387.504833] el1_abort+0x44/0x70
[ 4387.504840] el1h_64_sync_handler+0xd8/0xe4
[ 4387.504847] el1h_64_sync+0x64/0x68
[ 4387.504850] die_kernel_fault+0x1cc/0x364
[ 4387.504854] do_alignment_fault+0x0/0x80
[ 4387.504862] do_page_fault+0xb8/0x380
[ 4387.504869] do_translation_fault+0x9c/0xb0
[ 4387.504875] do_mem_abort+0x44/0x94
[ 4387.504882] el1_abort+0x44/0x70
[ 4387.504889] el1h_64_sync_handler+0xd8/0xe4
[ 4387.504893] el1h_64_sync+0x64/0x68
[ 4387.504899] die_kernel_fault+0x1cc/0x364
[ 4387.504903] do_alignment_fault+0x0/0x80
[ 4387.504908] do_page_fault+0xb8/0x380
[ 4387.504916] do_translation_fault+0x9c/0xb0
[ 4387.504921] do_mem_abort+0x44/0x94
[ 4387.504926] el1_abort+0x44/0x70
[ 4387.504933] el1h_64_sync_handler+0xd8/0xe4
[ 4387.504937] el1h_64_sync+0x64/0x68
[ 4387.504943] die_kernel_fault+0x1cc/0x364
[ 4387.504949] do_alignment_fault+0x0/0x80
[ 4387.504954] do_page_fault+0xb8/0x380
[ 4387.504962] do_translation_fault+0x9c/0xb0
[ 4387.504967] do_mem_abort+0x44/0x94
[ 4387.504973] el1_abort+0x44/0x70
[ 4387.504979] el1h_64_sync_handler+0xd8/0xe4
[ 4387.504984] el1h_64_sync+0x64/0x68
[ 4387.504987] die_kernel_fault+0x1cc/0x364
[ 4387.504991] do_alignment_fault+0x0/0x80
[ 4387.504997] do_page_fault+0xb8/0x380
[ 4387.505002] do_translation_fault+0x9c/0xb0
[ 4387.505007] do_mem_abort+0x44/0x94
[ 4387.505013] el1_abort+0x44/0x70
[ 4387.505017] el1h_64_sync_handler+0xd8/0xe4
[ 4387.505021] el1h_64_sync+0x64/0x68
[ 4387.505025] die_kernel_fault+0x1cc/0x364
[ 4387.505029] do_alignment_fault+0x0/0x80
[ 4387.505034] do_page_fault+0xb8/0x380
[ 4387.505040] do_translation_fault+0x9c/0xb0
[ 4387.505045] do_mem_abort+0x44/0x94
[ 4387.505050] el1_abort+0x44/0x70
[ 4387.505055] el1h_64_sync_handler+0xd8/0xe4
[ 4387.505059] el1h_64_sync+0x64/0x68
[ 4387.505063] die_kernel_fault+0x1cc/0x364
[ 4387.505067] do_alignment_fault+0x0/0x80
[ 4387.505072] do_page_fault+0xb8/0x380
[ 4387.505077] do_translation_fault+0x9c/0xb0
[ 4387.505085] do_mem_abort+0x44/0x94
[ 4387.505090] el1_abort+0x44/0x70
[ 4387.505095] el1h_64_sync_handler+0xd8/0xe4
[ 4387.505099] el1h_64_sync+0x64/0x68
[ 4387.505103] die_kernel_fault+0x1cc/0x364
[ 4387.505107] do_alignment_fault+0x0/0x80
[ 4387.505114] do_page_fault+0xb8/0x380
[ 4387.505119] do_translation_fault+0x9c/0xb0
[ 4387.505127] do_mem_abort+0x44/0x94
[ 4387.505135] el1_abort+0x44/0x70
[ 4387.505139] el1h_64_sync_handler+0xd8/0xe4
[ 4387.505144] el1h_64_sync+0x64/0x68
[ 4387.505147] die_kernel_fault+0x1cc/0x364
[ 4387.505151] do_alignment_fault+0x0/0x80
[ 4387.505156] do_page_fault+0xb8/0x380
[ 4387.505161] do_translation_fault+0x9c/0xb0
[ 4387.505167] do_mem_abort+0x44/0x94
[ 4387.505172] el1_abort+0x44/0x70
[ 4387.505179] el1h_64_sync_handler+0xd8/0xe4
[ 4387.505184] el1h_64_sync+0x64/0x68
[ 4387.505187] die_kernel_fault+0x1cc/0x364
[ 4387.505191] do_alignment_fault+0x0/0x80
[ 4387.505197] do_page_fault+0xb8/0x380
[ 4387.505202] do_translation_fault+0x9c/0xb0
[ 4387.505207] do_mem_abort+0x44/0x94
[ 4387.505215] el1_abort+0x44/0x70
[ 4387.505219] el1h_64_sync_handler+0xd8/0xe4
[ 4387.505224] el1h_64_sync+0x64/0x68
[ 4387.505230] die_kernel_fault+0x1cc/0x364
[ 4387.505234] do_alignment_fault+0x0/0x80
[ 4387.505239] do_page_fault+0xb8/0x380
[ 4387.505247] do_translation_fault+0x9c/0xb0
[ 4387.505252] do_mem_abort+0x44/0x94
[ 4387.505257] el1_abort+0x44/0x70
[ 4387.505262] el1h_64_sync_handler+0xd8/0xe4
[ 4387.505266] el1h_64_sync+0x64/0x68
[ 4387.505272] die_kernel_fault+0x1cc/0x364
[ 4387.505276] do_alignment_fault+0x0/0x80
[ 4387.505281] do_page_fault+0xb8/0x380
[ 4387.505286] do_translation_fault+0x9c/0xb0
[ 4387.505294] do_mem_abort+0x44/0x94
[ 4387.505300] el1_abort+0x44/0x70
[ 4387.505306] el1h_64_sync_handler+0xd8/0xe4
[ 4387.505311] el1h_64_sync+0x64/0x68
[ 4387.505314] spectre_v4_enable_task_mitigation+0x58/0x170
[ 4387.505321] __switch_to+0x90/0x170
[ 4387.505325] __schedule+0x2a4/0x8d0
[ 4387.505328] schedule+0x5c/0xd4
[ 4387.505332] smpboot_thread_fn+0xb0/0x320
[ 4387.505336] kthread+0x130/0x13c
[ 4387.505341] ret_from_fork+0x10/0x20
[ 1073.126275] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 1073.126286] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-3): P17/2:b..l [ 1073.126301] (detected by 0, t=84522 jiffies, g=9557, q=294 ncpus=4) [ 1073.126308] task:rcuc/0 state:S stack:0 pid:17 ppid:2 flags:0x00000008 [ 1073.126317] Call trace: [ 1073.126318] __switch_to+0xf4/0x170 [ 1073.126330] __schedule+0x2a4/0x8d0 [ 1073.126334] schedule+0x5c/0xd4 [ 1073.126338] smpboot_thread_fn+0xb0/0x320 [ 1073.126344] kthread+0x130/0x13c [ 1073.126350] ret_from_fork+0x10/0x20 [ 1073.126357] rcu: rcu_preempt kthread timer wakeup didn't happen for 19504 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 [ 1073.126361] rcu: Possible timer handling issue on cpu=0 timer-softirq=17448 [ 1073.126364] rcu: rcu_preempt kthread starved for 19505 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 [ 1073.126368] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 1073.126370] rcu: RCU grace-period kthread stack dump: [ 1073.126371] task:rcu_preempt state:I stack:0 pid:15 ppid:2 flags:0x00000008 [ 1073.126377] Call trace: [ 1073.126379] __switch_to+0xf4/0x170 [ 1073.126383] __schedule+0x2a4/0x8d0 [ 1073.126387] schedule+0x5c/0xd4 [ 1073.126390] schedule_timeout+0xa0/0x120 [ 1073.126396] rcu_gp_fqs_loop+0x1a0/0x740 [ 1073.126403] rcu_gp_kthread+0x254/0x310 [ 1073.126407] kthread+0x130/0x13c [ 1073.126412] ret_from_fork+0x10/0x20 [ 1073.126416] rcu: Stack dump where RCU GP kthread last ran: [ 1073.126420] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 6.1.55-rt16 #1 [ 1073.126425] Hardware name: LEAFF iMX8 M7 controller board (DT) [ 1073.126427] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 1073.126432] pc : cpuidle_enter_state+0xcc/0x380 [ 1073.126439] lr : cpuidle_enter_state+0xc0/0x380 [ 1073.126444] sp : ffff800009d23d00 [ 1073.126445] x29: ffff800009d23d00 x28: 0000000041d28288 x27: 00000000ffeced04 [ 1073.126451] x26: 0000000000000400 x25: 0000000000000001 x24: 000000f9db12954b [ 1073.126459] x23: 000000f9db4f31e5 x22: ffff0000d54c3080 x21: 0000000000000001 [ 1073.126470] x20: ffff0001f6f99e40 x19: 0000000000000001 x18: 0000000000000000 [ 1073.126478] x17: 000000040044ffff x16: 00500072b5503510 x15: 0000000000000000 [ 1073.126483] x14: ffff800009d41500 x13: ffff8001ed565000 x12: 0000000034d4d91d [ 1073.126489] x11: 0000000000000000 x10: 0000000000000000 x9 : ffff80000a702000 [ 1073.126499] x8 : ffff800009d41500 x7 : 00000000a998246c x6 : 0000000000013f48 [ 1073.126505] x5 : 0000000000000000 x4 : 0000000000000000 x3 : 0000000000000001 [ 1073.126510] x2 : 0000000000000001 x1 : 00000000000000e0 x0 : 000000000037d19b [ 1073.126518] Call trace: [ 1073.126519] cpuidle_enter_state+0xcc/0x380 [ 1073.126524] cpuidle_enter+0x38/0x50 [ 1073.126529] do_idle+0x270/0x310 [ 1073.126536] cpu_startup_entry+0x24/0x30 [ 1073.126539] rest_init+0x108/0x1a0 [ 1073.126545] arch_post_acpi_subsys_init+0x0/0x18 [ 1073.126552] start_kernel+0x70c/0x750 [ 1073.126557] __primary_switched+0xbc/0xc4 [ 1151.144720] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 1151.144733] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-3): P17/2:b..l [ 1151.144748] (detected by 0, t=104027 jiffies, g=9557, q=349 ncpus=4) [ 1151.144755] task:rcuc/0 state:S stack:0 pid:17 ppid:2 flags:0x00000008 [ 1151.144763] Call trace: [ 1151.144764] __switch_to+0xf4/0x170 [ 1151.144776] __schedule+0x2a4/0x8d0 [ 1151.144780] schedule+0x5c/0xd4 [ 1151.144784] smpboot_thread_fn+0xb0/0x320 [ 1151.144790] kthread+0x130/0x13c [ 1151.144796] ret_from_fork+0x10/0x20 [ 1151.144803] rcu: rcu_preempt kthread timer wakeup didn't happen for 19504 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 [ 1151.144808] rcu: Possible timer handling issue on cpu=0 timer-softirq=17448 [ 1151.144810] rcu: rcu_preempt kthread starved for 19505 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 [ 1151.144815] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 1151.144816] rcu: RCU grace-period kthread stack dump: [ 1151.144818] task:rcu_preempt state:I stack:0 pid:15 ppid:2 flags:0x00000008 [ 1151.144824] Call trace: [ 1151.144825] __switch_to+0xf4/0x170 [ 1151.144830] __schedule+0x2a4/0x8d0 [ 1151.144833] schedule+0x5c/0xd4 [ 1151.144837] schedule_timeout+0xa0/0x120 [ 1151.144843] rcu_gp_fqs_loop+0x1a0/0x740 [ 1151.144850] rcu_gp_kthread+0x254/0x310 [ 1151.144855] kthread+0x130/0x13c [ 1151.144859] ret_from_fork+0x10/0x20 [ 1151.144864] rcu: Stack dump where RCU GP kthread last ran: [ 1151.144868] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 6.1.55-rt16 #1 [ 1151.144872] Hardware name: LEAFF iMX8 M7 controller board (DT) [ 1151.144875] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 1151.144880] pc : cpuidle_enter_state+0xcc/0x380 [ 1151.144887] lr : cpuidle_enter_state+0xc0/0x380 [ 1151.144892] sp : ffff800009d23d00 [ 1151.144893] x29: ffff800009d23d00 x28: 0000000041d28288 x27: 00000000ffeced04 [ 1151.144900] x26: 0000000000000400 x25: 0000000000000001 x24: 0000010c05559607 [ 1151.144905] x23: 0000010c05923495 x22: ffff0000d54c3080 x21: 0000000000000001 [ 1151.144913] x20: ffff0001f6f99e40 x19: 0000000000000001 x18: 0000000000000000 [ 1151.144921] x17: 000000040044ffff x16: 00500072b5503510 x15: 0000000000000000 [ 1151.144929] x14: ffff800009d41500 x13: ffff8001ed565000 x12: 0000000034d4d91d [ 1151.144935] x11: 0000000000000000 x10: 0000000000000000 x9 : ffff80000a702000 [ 1151.144940] x8 : ffff800009d41500 x7 : 00000000a998246c x6 : 0000000000013f48 [ 1151.144948] x5 : 0000000000000000 x4 : 0000000000000000 x3 : 0000000000000001 [ 1151.144953] x2 : 0000000000000001 x1 : 00000000000000e0 x0 : 0000000000390273 [ 1151.144959] Call trace: [ 1151.144960] cpuidle_enter_state+0xcc/0x380 [ 1151.144965] cpuidle_enter+0x38/0x50 [ 1151.144970] do_idle+0x270/0x310 [ 1151.144977] cpu_startup_entry+0x28/0x30 [ 1151.144980] rest_init+0x108/0x1a0 [ 1151.144986] arch_post_acpi_subsys_init+0x0/0x18 [ 1151.144993] start_kernel+0x70c/0x750 [ 1151.144998] __primary_switched+0xbc/0xc4 [ 1229.163167] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 1229.163179] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-3): P17/2:b..l [ 1229.163195] (detected by 0, t=123532 jiffies, g=9557, q=405 ncpus=4) [ 1229.163202] task:rcuc/0 state:S stack:0 pid:17 ppid:2 flags:0x00000008 [ 1229.163210] Call trace: [ 1229.163211] __switch_to+0xf4/0x170 [ 1229.163223] __schedule+0x2a4/0x8d0 [ 1229.163228] schedule+0x5c/0xd4 [ 1229.163231] smpboot_thread_fn+0xb0/0x320 [ 1229.163237] kthread+0x130/0x13c [ 1229.163243] ret_from_fork+0x10/0x20 [ 1229.163250] rcu: rcu_preempt kthread timer wakeup didn't happen for 19504 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 [ 1229.163255] rcu: Possible timer handling issue on cpu=0 timer-softirq=17448 [ 1229.163257] rcu: rcu_preempt kthread starved for 19505 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 [ 1229.163262] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 1229.163264] rcu: RCU grace-period kthread stack dump: [ 1229.163265] task:rcu_preempt state:I stack:0 pid:15 ppid:2 flags:0x00000008 [ 1229.163271] Call trace: [ 1229.163273] __switch_to+0xf4/0x170 [ 1229.163277] __schedule+0x2a4/0x8d0 [ 1229.163281] schedule+0x5c/0xd4 [ 1229.163284] schedule_timeout+0xa0/0x120 [ 1229.163290] rcu_gp_fqs_loop+0x1a0/0x740 [ 1229.163298] rcu_gp_kthread+0x254/0x310 [ 1229.163301] kthread+0x130/0x13c [ 1229.163306] ret_from_fork+0x10/0x20 [ 1229.163310] rcu: Stack dump where RCU GP kthread last ran: [ 1229.163314] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 6.1.55-rt16 #1 [ 1229.163318] Hardware name: LEAFF iMX8 M7 controller board (DT) [ 1229.163321] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 1229.163325] pc : cpuidle_enter_state+0xcc/0x380 [ 1229.163333] lr : cpuidle_enter_state+0xc0/0x380 [ 1229.163337] sp : ffff800009d23d00 [ 1229.163339] x29: ffff800009d23d00 x28: 0000000041d28288 x27: 00000000ffeced04 [ 1229.163345] x26: 0000000000000400 x25: 0000000000000001 x24: 0000011e2f989c22 [ 1229.163351] x23: 0000011e2fd53ab0 x22: ffff0000d54c3080 x21: 0000000000000001 [ 1229.163356] x20: ffff0001f6f99e40 x19: 0000000000000001 x18: 0000000000000000 [ 1229.163362] x17: 000000040044ffff x16: 00500072b5503510 x15: 0000000000000000 [ 1229.163372] x14: ffff800009d41500 x13: ffff8001ed565000 x12: 0000000034d4d91d [ 1229.163380] x11: 0000000000000000 x10: 0000000000000000 x9 : ffff80000a702000 [ 1229.163386] x8 : ffff800009d41500 x7 : 00000000a998246c x6 : 0000000000013f48 [ 1229.163393] x5 : 0000000000000000 x4 : 0000000000000000 x3 : 0000000000000001 [ 1229.163401] x2 : 0000000000000001 x1 : 00000000000000e0 x0 : 00000000003a333f [ 1229.163406] Call trace: [ 1229.163407] cpuidle_enter_state+0xcc/0x380 [ 1229.163413] cpuidle_enter+0x38/0x50 [ 1229.163417] do_idle+0x270/0x310 [ 1229.163425] cpu_startup_entry+0x28/0x30 [ 1229.163428] rest_init+0x108/0x1a0 [ 1229.163434] arch_post_acpi_subsys_init+0x0/0x18 [ 1229.163441] start_kernel+0x70c/0x750 [ 1229.163447] __primary_switched+0xbc/0xc4 [ 1247.450882] BUG: spinlock bad magic on CPU#1, ktimers/1/25 [ 1247.450894] lock: 0xffff0001f6fa8a08, .magic: 00000000, .owner: <none>/-1, .owner_cpu: 0 [ 1247.450908] CPU: 1 PID: 25 Comm: ktimers/1 Tainted: G W 6.1.55-rt16 #1 [ 1247.450913] Hardware name: LEAFF iMX8 M7 controller board (DT) [ 1247.450917] Call trace: [ 1247.450918] dump_backtrace.part.0+0xe4/0xf0 [ 1247.450930] show_stack+0x18/0x30 [ 1247.450936] dump_stack_lvl+0x88/0xb4 [ 1247.450943] dump_stack+0x18/0x34 [ 1247.450946] spin_dump+0x84/0x98 [ 1247.450952] do_raw_spin_lock+0x108/0x130 [ 1247.450959] _raw_spin_lock_irqsave+0x78/0xb0 [ 1247.450965] rt_spin_lock+0x64/0x10c [ 1247.450970] __run_timers+0x60/0x3c0 [ 1247.450977] run_timer_softirq+0x20/0x40 [ 1247.450982] _stext+0x124/0x494 [ 1247.450986] run_timersd+0x70/0xd4 [ 1247.450993] smpboot_thread_fn+0x220/0x320 [ 1247.450997] kthread+0x130/0x13c [ 1247.451003] ret_from_fork+0x10/0x20 [ 1307.181302] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 1307.181311] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-3): P17/2:b..l [ 1307.181324] (detected by 0, t=143037 jiffies, g=9557, q=480 ncpus=4) [ 1307.181330] task:rcuc/0 state:S stack:0 pid:17 ppid:2 flags:0x00000008 [ 1307.181337] Call trace: [ 1307.181339] __switch_to+0xf4/0x170 [ 1307.181350] __schedule+0x2a4/0x8d0 [ 1307.181355] schedule+0x5c/0xd4 [ 1307.181358] smpboot_thread_fn+0xb0/0x320 [ 1307.181364] kthread+0x130/0x13c [ 1307.181370] ret_from_fork+0x10/0x20 [ 1307.181377] rcu: rcu_preempt kthread timer wakeup didn't happen for 19504 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 [ 1307.181382] rcu: Possible timer handling issue on cpu=0 timer-softirq=17448 [ 1307.181384] rcu: rcu_preempt kthread starved for 19505 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 [ 1307.181389] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 1307.181390] rcu: RCU grace-period kthread stack dump: [ 1307.181392] task:rcu_preempt state:I stack:0 pid:15 ppid:2 flags:0x00000008 [ 1307.181397] Call trace: [ 1307.181399] __switch_to+0xf4/0x170 [ 1307.181403] __schedule+0x2a4/0x8d0 [ 1307.181406] schedule+0x5c/0xd4 [ 1307.181410] schedule_timeout+0xa0/0x120 [ 1307.181416] rcu_gp_fqs_loop+0x1a0/0x740 [ 1307.181423] rcu_gp_kthread+0x254/0x310 [ 1307.181427] kthread+0x130/0x13c [ 1307.181431] ret_from_fork+0x10/0x20 [ 1307.181435] rcu: Stack dump where RCU GP kthread last ran: [ 1307.181439] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 6.1.55-rt16 #1 [ 1307.181444] Hardware name: LEAFF iMX8 M7 controller board (DT) [ 1307.181446] pstate: 00000005 (nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 1307.181451] pc : cpuidle_enter_state+0xcc/0x380 [ 1307.181458] lr : cpuidle_enter_state+0xc0/0x380 [ 1307.181463] sp : ffff800009d23d00 [ 1307.181465] x29: ffff800009d23d00 x28: 0000000041d28288 x27: 00000000ffeced04 [ 1307.181471] x26: 0000000000000400 x25: 0000000000000001 x24: 0000013059d6d82a [ 1307.181481] x23: 000001305a138f22 x22: ffff0000d54c3080 x21: 0000000000000001 [ 1307.181487] x20: ffff0001f6f99e40 x19: 0000000000000001 x18: 0000000000000000 [ 1307.181492] x17: 000000040044ffff x16: 00500072b5503510 x15: 0000000000000000 [ 1307.181500] x14: ffff800009d41500 x13: ffff8001ed565000 x12: 0000000034d4d91d [ 1307.181511] x11: 0000000000000000 x10: 0000000000001000 x9 : ffff800008010800 [ 1307.181516] x8 : ffff800009a23008 x7 : 00000072b5503510 x6 : 0000000271b7ff54 [ 1307.181522] x5 : 0000000000000000 x4 : ffff800009d1ffff x3 : 0000000000000000 [ 1307.181529] x2 : 0000000100000001 x1 : 00000000000000e0 x0 : ffff800008cd4280 [ 1307.181537] Call trace: [ 1307.181539] cpuidle_enter_state+0xcc/0x380 [ 1307.181544] cpuidle_enter+0x38/0x50 [ 1307.181548] do_idle+0x270/0x310 [ 1307.181555] cpu_startup_entry+0x28/0x30 [ 1307.181559] rest_init+0x108/0x1a0 [ 1307.181565] arch_post_acpi_subsys_init+0x0/0x18 [ 1307.181572] start_kernel+0x70c/0x750 [ 1307.181577] __primary_switched+0xbc/0xc4 [ 1385.200027] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 1385.200038] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-3): P17/2:b..l [ 1385.200051] (detected by 0, t=162542 jiffies, g=9557, q=535 ncpus=4) [ 1385.200058] task:rcuc/0 state:S stack:0 pid:17 ppid:2 flags:0x00000008 [ 1385.200064] Call trace: [ 1385.200066] __switch_to+0xf4/0x170 [ 1385.200078] __schedule+0x2a4/0x8d0 [ 1385.200082] schedule+0x5c/0xd4 [ 1385.200086] smpboot_thread_fn+0xb0/0x320 [ 1385.200092] kthread+0x130/0x13c [ 1385.200097] ret_from_fork+0x10/0x20 [ 1385.200104] rcu: rcu_preempt kthread timer wakeup didn't happen for 19504 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 [ 1385.200109] rcu: Possible timer handling issue on cpu=0 timer-softirq=17448 [ 1385.200111] rcu: rcu_preempt kthread starved for 19505 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 [ 1385.200116] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 1385.200117] rcu: RCU grace-period kthread stack dump: [ 1385.200119] task:rcu_preempt state:I stack:0 pid:15 ppid:2 flags:0x00000008 [ 1385.200124] Call trace: [ 1385.200125] __switch_to+0xf4/0x170 [ 1385.200129] __schedule+0x2a4/0x8d0 [ 1385.200133] schedule+0x5c/0xd4 [ 1385.200137] schedule_timeout+0xa0/0x120 [ 1385.200142] rcu_gp_fqs_loop+0x1a0/0x740 [ 1385.200150] rcu_gp_kthread+0x254/0x310 [ 1385.200153] kthread+0x130/0x13c [ 1385.200158] ret_from_fork+0x10/0x20 [ 1385.200162] rcu: Stack dump where RCU GP kthread last ran: [ 1385.200166] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 6.1.55-rt16 #1 [ 1385.200170] Hardware name: LEAFF iMX8 M7 controller board (DT) [ 1385.200173] pstate: 00000005 (nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 1385.200178] pc : cpuidle_enter_state+0xcc/0x380 [ 1385.200184] lr : cpuidle_enter_state+0xc0/0x380 [ 1385.200189] sp : ffff800009d23d00 [ 1385.200191] x29: ffff800009d23d00 x28: 0000000041d28288 x27: 00000000ffeced04 [ 1385.200197] x26: 0000000000000400 x25: 0000000000000001 x24: 00000142841e2ec3 [ 1385.200205] x23: 00000142845ae250 x22: ffff0000d54c3080 x21: 0000000000000001 [ 1385.200215] x20: ffff0001f6f99e40 x19: 0000000000000001 x18: 0000000000000000 [ 1385.200221] x17: 000000040044ffff x16: 00500072b5503510 x15: 0000000000000000 [ 1385.200227] x14: ffff800009d41500 x13: ffff8001ed565000 x12: 0000000034d4d91d [ 1385.200235] x11: 0000000000000000 x10: 0000000000001000 x9 : ffff800008010800 [ 1385.200245] x8 : ffff800009a23008 x7 : 00000072b5503510 x6 : ffff800008136528 [ 1385.200250] x5 : 0000000000000000 x4 : ffff800009d1ffff x3 : 0000000000000000 [ 1385.200256] x2 : 0000000100000001 x1 : 00000000000000e0 x0 : ffff800008cd4280 [ 1385.200266] Call trace: [ 1385.200267] cpuidle_enter_state+0xcc/0x380 [ 1385.200272] cpuidle_enter+0x38/0x50 [ 1385.200277] do_idle+0x270/0x310 [ 1385.200284] cpu_startup_entry+0x24/0x30 [ 1385.200287] rest_init+0x108/0x1a0 [ 1385.200293] arch_post_acpi_subsys_init+0x0/0x18 [ 1385.200300] start_kernel+0x70c/0x750 [ 1385.200305] __primary_switched+0xbc/0xc4 [ 1463.218479] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 1463.218488] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-3): P17/2:b..l [ 1463.218501] (detected by 0, t=182047 jiffies, g=9557, q=590 ncpus=4) [ 1463.218507] task:rcuc/0 state:S stack:0 pid:17 ppid:2 flags:0x00000008 [ 1463.218514] Call trace: [ 1463.218516] __switch_to+0xf4/0x170 [ 1463.218527] __schedule+0x2a4/0x8d0 [ 1463.218531] schedule+0x5c/0xd4 [ 1463.218535] smpboot_thread_fn+0xb0/0x320 [ 1463.218541] kthread+0x130/0x13c [ 1463.218547] ret_from_fork+0x10/0x20 [ 1463.218554] rcu: rcu_preempt kthread timer wakeup didn't happen for 19504 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 [ 1463.218558] rcu: Possible timer handling issue on cpu=0 timer-softirq=17448 [ 1463.218560] rcu: rcu_preempt kthread starved for 19505 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 [ 1463.218565] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 1463.218567] rcu: RCU grace-period kthread stack dump: [ 1463.218568] task:rcu_preempt state:I stack:0 pid:15 ppid:2 flags:0x00000008 [ 1463.218573] Call trace: [ 1463.218574] __switch_to+0xf4/0x170 [ 1463.218578] __schedule+0x2a4/0x8d0 [ 1463.218582] schedule+0x5c/0xd4 [ 1463.218586] schedule_timeout+0xa0/0x120 [ 1463.218592] rcu_gp_fqs_loop+0x1a0/0x740 [ 1463.218600] rcu_gp_kthread+0x254/0x310 [ 1463.218603] kthread+0x130/0x13c [ 1463.218608] ret_from_fork+0x10/0x20 [ 1463.218612] rcu: Stack dump where RCU GP kthread last ran: [ 1463.218616] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 6.1.55-rt16 #1 [ 1463.218620] Hardware name: LEAFF iMX8 M7 controller board (DT) [ 1463.218622] pstate: 00000005 (nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 1463.218627] pc : cpuidle_enter_state+0xcc/0x380 [ 1463.218634] lr : cpuidle_enter_state+0xc0/0x380 [ 1463.218639] sp : ffff800009d23d00 [ 1463.218640] x29: ffff800009d23d00 x28: 0000000041d28288 x27: 00000000ffeced04 [ 1463.218646] x26: 0000000000000400 x25: 0000000000000001 x24: 00000154ae614384 [ 1463.218654] x23: 00000154ae9df9ff x22: ffff0000d54c3080 x21: 0000000000000001 [ 1463.218662] x20: ffff0001f6f99e40 x19: 0000000000000001 x18: 0000000000000000 [ 1463.218672] x17: 000000040044ffff x16: 00500072b5503510 x15: 0000000000000000 [ 1463.218678] x14: ffff800009d41500 x13: ffff8001ed565000 x12: 0000000034d4d91d [ 1463.218684] x11: 0000000000000000 x10: 0000000000001000 x9 : ffff800008010800 [ 1463.218692] x8 : ffff800009a23008 x7 : 00000072b5503510 x6 : ffff800008136528 [ 1463.218702] x5 : 0000000000000000 x4 : ffff800009d1ffff x3 : 0000000000000000 [ 1463.218707] x2 : 0000000100000001 x1 : 00000000000000e0 x0 : ffff800008cd4280 [ 1463.218713] Call trace: [ 1463.218714] cpuidle_enter_state+0xcc/0x380 [ 1463.218719] cpuidle_enter+0x38/0x50 [ 1463.218724] do_idle+0x270/0x310 [ 1463.218731] cpu_startup_entry+0x28/0x30 [ 1463.218734] rest_init+0x108/0x1a0 [ 1463.218740] arch_post_acpi_subsys_init+0x0/0x18 [ 1463.218747] start_kernel+0x70c/0x750 [ 1463.218752] __primary_switched+0xbc/0xc4 [ 1541.236914] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 1541.236922] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-3): P17/2:b..l [ 1541.236934] (detected by 0, t=201552 jiffies, g=9557, q=643 ncpus=4) [ 1541.236940] task:rcuc/0 state:S stack:0 pid:17 ppid:2 flags:0x00000008 [ 1541.236947] Call trace: [ 1541.236948] __switch_to+0xf4/0x170 [ 1541.236959] __schedule+0x2a4/0x8d0 [ 1541.236963] schedule+0x5c/0xd4 [ 1541.236967] smpboot_thread_fn+0xb0/0x320 [ 1541.236973] kthread+0x130/0x13c [ 1541.236979] ret_from_fork+0x10/0x20 [ 1541.236986] rcu: rcu_preempt kthread timer wakeup didn't happen for 19504 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 [ 1541.236990] rcu: Possible timer handling issue on cpu=0 timer-softirq=17448 [ 1541.236993] rcu: rcu_preempt kthread starved for 19505 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 [ 1541.236997] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 1541.236999] rcu: RCU grace-period kthread stack dump: [ 1541.237000] task:rcu_preempt state:I stack:0 pid:15 ppid:2 flags:0x00000008 [ 1541.237005] Call trace: [ 1541.237007] __switch_to+0xf4/0x170 [ 1541.237011] __schedule+0x2a4/0x8d0 [ 1541.237015] schedule+0x5c/0xd4 [ 1541.237018] schedule_timeout+0xa0/0x120 [ 1541.237025] rcu_gp_fqs_loop+0x1a0/0x740 [ 1541.237032] rcu_gp_kthread+0x254/0x310 [ 1541.237036] kthread+0x130/0x13c [ 1541.237040] ret_from_fork+0x10/0x20 [ 1541.237044] rcu: Stack dump where RCU GP kthread last ran: [ 1541.237048] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 6.1.55-rt16 #1 [ 1541.237053] Hardware name: LEAFF iMX8 M7 controller board (DT) [ 1541.237055] pstate: 00000005 (nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 1541.237060] pc : cpuidle_enter_state+0xcc/0x380 [ 1541.237067] lr : cpuidle_enter_state+0xc0/0x380 [ 1541.237072] sp : ffff800009d23d00 [ 1541.237073] x29: ffff800009d23d00 x28: 0000000041d28288 x27: 00000000ffeced04 [ 1541.237079] x26: 0000000000000400 x25: 0000000000000001 x24: 00000166d8a3e9eb [ 1541.237085] x23: 00000166d8e0cae1 x22: ffff0000d54c3080 x21: 0000000000000001 [ 1541.237091] x20: ffff0001f6f99e40 x19: 0000000000000001 x18: 0000000000000000 [ 1541.237099] x17: 000000040044ffff x16: 00500072b5503510 x15: 0000000000000000 [ 1541.237109] x14: ffff800009d41500 x13: ffff8001ed565000 x12: 0000000034d4d91d [ 1541.237115] x11: 0000000000000000 x10: 0000000000001000 x9 : ffff800008010800 [ 1541.237120] x8 : ffff800009a23008 x7 : 00000072b5503510 x6 : ffff800008136528 [ 1541.237128] x5 : 0000000000000000 x4 : ffff800009d1ffff x3 : 0000000000000000 [ 1541.237136] x2 : 0000000100000001 x1 : 00000000000000e0 x0 : ffff800008cd4280 [ 1541.237142] Call trace: [ 1541.237143] cpuidle_enter_state+0xcc/0x380 [ 1541.237148] cpuidle_enter+0x38/0x50 [ 1541.237153] do_idle+0x270/0x310 [ 1541.237159] cpu_startup_entry+0x24/0x30 [ 1541.237163] rest_init+0x108/0x1a0 [ 1541.237168] arch_post_acpi_subsys_init+0x0/0x18 [ 1541.237175] start_kernel+0x70c/0x750 [ 1541.237181] __primary_switched+0xbc/0xc4 [ 1619.255349] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 1619.255358] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-3): P17/2:b..l [ 1619.255371] (detected by 0, t=221057 jiffies, g=9557, q=698 ncpus=4) [ 1619.255377] task:rcuc/0 state:S stack:0 pid:17 ppid:2 flags:0x00000008 [ 1619.255384] Call trace: [ 1619.255386] __switch_to+0xf4/0x170 [ 1619.255397] __schedule+0x2a4/0x8d0 [ 1619.255402] schedule+0x5c/0xd4 [ 1619.255406] smpboot_thread_fn+0xb0/0x320 [ 1619.255411] kthread+0x130/0x13c [ 1619.255417] ret_from_fork+0x10/0x20 [ 1619.255424] rcu: rcu_preempt kthread timer wakeup didn't happen for 19504 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 [ 1619.255429] rcu: Possible timer handling issue on cpu=0 timer-softirq=17448 [ 1619.255431] rcu: rcu_preempt kthread starved for 19505 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 [ 1619.255435] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 1619.255437] rcu: RCU grace-period kthread stack dump: [ 1619.255439] task:rcu_preempt state:I stack:0 pid:15 ppid:2 flags:0x00000008 [ 1619.255444] Call trace: [ 1619.255445] __switch_to+0xf4/0x170 [ 1619.255449] __schedule+0x2a4/0x8d0 [ 1619.255453] schedule+0x5c/0xd4 [ 1619.255457] schedule_timeout+0xa0/0x120 [ 1619.255462] rcu_gp_fqs_loop+0x1a0/0x740 [ 1619.255470] rcu_gp_kthread+0x254/0x310 [ 1619.255474] kthread+0x130/0x13c [ 1619.255478] ret_from_fork+0x10/0x20 [ 1619.255482] rcu: Stack dump where RCU GP kthread last ran: [ 1619.255486] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 6.1.55-rt16 #1 [ 1619.255490] Hardware name: LEAFF iMX8 M7 controller board (DT) [ 1619.255493] pstate: 00000005 (nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 1619.255497] pc : cpuidle_enter_state+0xcc/0x380 [ 1619.255504] lr : cpuidle_enter_state+0xc0/0x380 [ 1619.255509] sp : ffff800009d23d00 [ 1619.255510] x29: ffff800009d23d00 x28: 0000000041d28288 x27: 00000000ffeced04 [ 1619.255517] x26: 0000000000000400 x25: 0000000000000001 x24: 0000017902e6e25a [ 1619.255527] x23: 000001790323a96f x22: ffff0000d54c3080 x21: 0000000000000001 [ 1619.255532] x20: ffff0001f6f99e40 x19: 0000000000000001 x18: 0000000000000000 [ 1619.255538] x17: 000000040044ffff x16: 00500072b5503510 x15: 0000000000000000 [ 1619.255546] x14: ffff800009d41500 x13: ffff8001ed565000 x12: 0000000034d4d91d [ 1619.255556] x11: 0000000000000000 x10: 0000000000001000 x9 : ffff800008010800 [ 1619.255562] x8 : ffff800009a23008 x7 : 00000072b5503510 x6 : ffff800008136528 [ 1619.255567] x5 : 0000000000000000 x4 : ffff800009d1ffff x3 : 0000000000000000 [ 1619.255575] x2 : 0000000100000001 x1 : 00000000000000e0 x0 : ffff800008cd4280 [ 1619.255585] Call trace: [ 1619.255586] cpuidle_enter_state+0xcc/0x380 [ 1619.255591] cpuidle_enter+0x38/0x50 [ 1619.255596] do_idle+0x270/0x310 [ 1619.255603] cpu_startup_entry+0x28/0x30 [ 1619.255606] rest_init+0x108/0x1a0 [ 1619.255612] arch_post_acpi_subsys_init+0x0/0x18 [ 1619.255619] start_kernel+0x70c/0x750 [ 1619.255624] __primary_switched+0xbc/0xc4 [ 1697.273813] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: [ 1697.273822] rcu: Tasks blocked on level-0 rcu_node (CPUs 0-3): P17/2:b..l [ 1697.273834] (detected by 0, t=240562 jiffies, g=9557, q=752 ncpus=4) [ 1697.273840] task:rcuc/0 state:S stack:0 pid:17 ppid:2 flags:0x00000008 [ 1697.273847] Call trace: [ 1697.273848] __switch_to+0xf4/0x170 [ 1697.273859] __schedule+0x2a4/0x8d0 [ 1697.273864] schedule+0x5c/0xd4 [ 1697.273867] smpboot_thread_fn+0xb0/0x320 [ 1697.273873] kthread+0x130/0x13c [ 1697.273879] ret_from_fork+0x10/0x20 [ 1697.273886] rcu: rcu_preempt kthread timer wakeup didn't happen for 19504 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 [ 1697.273890] rcu: Possible timer handling issue on cpu=0 timer-softirq=17448 [ 1697.273892] rcu: rcu_preempt kthread starved for 19505 jiffies! g9557 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0 [ 1697.273897] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. [ 1697.273898] rcu: RCU grace-period kthread stack dump: [ 1697.273900] task:rcu_preempt state:I stack:0 pid:15 ppid:2 flags:0x00000008 [ 1697.273905] Call trace: [ 1697.273907] __switch_to+0xf4/0x170 [ 1697.273911] __schedule+0x2a4/0x8d0 [ 1697.273915] schedule+0x5c/0xd4 [ 1697.273918] schedule_timeout+0xa0/0x120 [ 1697.273924] rcu_gp_fqs_loop+0x1a0/0x740 [ 1697.273931] rcu_gp_kthread+0x254/0x310 [ 1697.273935] kthread+0x130/0x13c [ 1697.273939] ret_from_fork+0x10/0x20 [ 1697.273944] rcu: Stack dump where RCU GP kthread last ran: [ 1697.273948] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 6.1.55-rt16 #1 [ 1697.273953] Hardware name: LEAFF iMX8 M7 controller board (DT) [ 1697.273955] pstate: 00000005 (nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 1697.273960] pc : cpuidle_enter_state+0xcc/0x380 [ 1697.273967] lr : cpuidle_enter_state+0xc0/0x380 [ 1697.273971] sp : ffff800009d23d00 [ 1697.273973] x29: ffff800009d23d00 x28: 0000000041d28288 x27: 00000000ffeced04 [ 1697.273979] x26: 0000000000000400 x25: 0000000000000001 x24: 0000018b2d29f3b0 [ 1697.273987] x23: 0000018b2d66e82e x22: ffff0000d54c3080 x21: 0000000000000001 [ 1697.273997] x20: ffff0001f6f99e40 x19: 0000000000000001 x18: 0000000000000000 [ 1697.274003] x17: 000000040044ffff x16: 00500072b5503510 x15: 0000000000000000 [ 1697.274009] x14: ffff800009d41500 x13: ffff8001ed565000 x12: 0000000034d4d91d [ 1697.274017] x11: 0000000000000000 x10: 0000000000001000 x9 : ffff800008010800 [ 1697.274027] x8 : ffff800009a23008 x7 : 00000072b5503510 x6 : ffff800008136528 [ 1697.274033] x5 : 0000000000000000 x4 : ffff800009d1ffff x3 : 0000000000000000 [ 1697.274038] x2 : 0000000100000001 x1 : 00000000000000e0 x0 : ffff800008cd4280 [ 1697.274049] Call trace: [ 1697.274050] cpuidle_enter_state+0xcc/0x380 [ 1697.274055] cpuidle_enter+0x38/0x50 [ 1697.274060] do_idle+0x270/0x310 [ 1697.274066] cpu_startup_entry+0x24/0x30 [ 1697.274070] rest_init+0x108/0x1a0 [ 1697.274076] arch_post_acpi_subsys_init+0x0/0x18 [ 1697.274082] start_kernel+0x70c/0x750 [ 1697.274088] __primary_switched+0xbc/0xc4
