Thanks for conformation. Can you please try once with the below patch which I have posted today https://patchwork.kernel.org/patch/10819769/ Thanks, Sreekanth -----Original Message----- From: Michal Soltys [mailto:soltys@xxxxxxxx] Sent: Monday, February 18, 2019 7:09 PM To: Sreekanth Reddy <sreekanth.reddy@xxxxxxxxxxxx>; linux-scsi@xxxxxxxxxxxxxxx Subject: Re: kernel panic in late shutdown stage (after syncing) in _scsih_set_satl_pending() in mpt3sas On 2/18/19 11:09 AM, Sreekanth Reddy wrote: > Hi Michal, > > This issue looks to be same as issue discussed in below URL thread, > https://patchwork.kernel.org/patch/10734933/ > > This issue happens only when MQ is enabled, can you please try once > with disabling the MQ? > > Thanks, > Sreekanth Yes, no issues with mq disabled. I also tested that v2 patch in mq mode - no kernel panics with it either, so indeed it seems it's the same issue. > > -----Original Message----- > From: linux-scsi-owner@xxxxxxxxxxxxxxx > [mailto:linux-scsi-owner@xxxxxxxxxxxxxxx] On Behalf Of Michal Soltys > Sent: Friday, February 15, 2019 9:36 PM > To: linux-scsi@xxxxxxxxxxxxxxx > Subject: kernel panic in late shutdown stage (after syncing) in > _scsih_set_satl_pending() in mpt3sas > > Hi, > > This basically happens on most shutdowns (and one of the ways to ensure it > happens always is to for example change i/o schedulers after the boot via > simple udev rules). > > Normally (on successful shutdown), the very last things displayed right > after sync messages are "sending message unit reset !!", followed by > "message unit reset: success". > > On kernel panic, the following trace was captured via serial console > (tested > with recent 4.19 and 4.20 kernels): > > [ 41.563811] systemd-shutdown[1]: All filesystems unmounted. > [ 41.591078] systemd-shutdown[1]: Deactivating swaps. > [ 41.618557] systemd-shutdown[1]: All swaps deactivated. > [ 41.646348] systemd-shutdown[1]: Detaching loop devices. > [ 41.673201] systemd-shutdown[1]: All loop devices detached. > [ 41.700853] systemd-shutdown[1]: Detaching DM devices. > [ 42.765286] kvm: exiting hardware virtualization > [ 42.790096] sd 2:0:0:0: [sds] Synchronizing SCSI cache > [ 42.814967] sd 2:0:0:0: [sds] Stopping disk > [ 43.585858] sd 0:0:16:0: [sdr] Synchronizing SCSI cache > [ 43.610431] sd 0:0:15:0: [sdq] Synchronizing SCSI cache > [ 43.635195] sd 0:0:14:0: [sdp] Synchronizing SCSI cache > [ 43.659712] sd 0:0:13:0: [sdo] Synchronizing SCSI cache > [ 43.684316] sd 0:0:12:0: [sdn] Synchronizing SCSI cache > [ 43.708725] sd 0:0:11:0: [sdm] Synchronizing SCSI cache > [ 43.732968] sd 0:0:10:0: [sdl] Synchronizing SCSI cache > [ 43.756784] sd 0:0:9:0: [sdk] Synchronizing SCSI cache > [ 43.780457] sd 0:0:8:0: [sdj] Synchronizing SCSI cache > [ 43.804158] sd 0:0:7:0: [sdi] Synchronizing SCSI cache > [ 43.887202] sd 0:0:6:0: [sdh] Synchronizing SCSI cache > [ 43.910297] sd 0:0:5:0: [sdg] Synchronizing SCSI cache > [ 43.933208] sd 0:0:4:0: [sdf] Synchronizing SCSI cache > [ 43.960025] sd 0:0:3:0: [sde] Synchronizing SCSI cache > [ 43.986534] sd 0:0:2:0: [sdd] Synchronizing SCSI cache > [ 44.012309] sd 0:0:1:0: [sdc] Synchronizing SCSI cache > [ 44.038061] sd 0:0:0:0: [sdb] Synchronizing SCSI cache > [ 44.059414] sd 1:0:0:0: [sda] Synchronizing SCSI cache > [ 44.082177] sd 1:0:0:0: [sda] Stopping disk > [ 45.131598] general protection fault: 0000 [#1] SMP NOPTI > [ 45.152880] CPU: 4 PID: 1 Comm: systemd-shutdow Tainted: G E > 4.20.7 #2 > [ 45.176893] Hardware name: Supermicro X9DR7/E-(J)LN4F/X9DR7/E-(J)LN4F, > BIOS 3.3 07/13/2018 > [ 45.201312] RIP: 0010:_scsih_set_satl_pending+0xc/0x50 [mpt3sas] > [ 45.224274] Code: c3 48 c1 ee 0b b8 20 00 00 00 ba 40 00 00 00 89 41 04 > 89 11 31 c0 89 71 08 c3 0f 1f 40 00 0f 1f 44 00 00 48 8b 87 f8 00 00 00 > <0f> > b6 10 80 fa a1 74 09 31 c0 80 fa 85 74 02 f3 c3 48 8b 47 38 40 > [ 45.279464] RSP: 0018:ffffbb31000bfd58 EFLAGS: 00010202 > [ 45.303230] RAX: 5d8824840b490000 RBX: ffff9179613c47b8 RCX: > 0000000000000000 > [ 45.330342] RDX: ffff9179613c4680 RSI: 0000000000000000 RDI: > ffff9179613c47b8 > [ 45.358839] RBP: 0000000000000d48 R08: 0000000000000002 R09: > ffffbb31000bfd24 > [ 45.385354] R10: 0000000000000000 R11: 0000000000002eaf R12: > ffff91796d4d67a8 > [ 45.411901] R13: 0000000000000001 R14: ffff91796cd87110 R15: > 0000000000000000 > [ 45.438156] FS: 00007f59cdc31900(0000) GS:ffff917d6fa00000(0000) > knlGS:0000000000000000 > [ 45.465301] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 45.490454] CR2: 00007f3caf6258dc CR3: 000000046b4f0006 CR4: > 00000000001606e0 > [ 45.517481] Call Trace: > [ 45.539085] _scsih_flush_running_cmds+0x41/0xc0 [mpt3sas] > [ 45.567700] scsih_shutdown+0x37/0xb0 [mpt3sas] > [ 45.594390] pci_device_shutdown+0x34/0x60 > [ 45.619163] device_shutdown+0x15d/0x210 > [ 45.644653] kernel_power_off+0x31/0x70 > [ 45.668895] __do_sys_reboot+0x18f/0x210 > [ 45.694431] ? do_sigtimedwait+0x1ec/0x220 > [ 45.718983] ? do_writev+0x5e/0xf0 > [ 45.740803] ? do_writev+0x5e/0xf0 > [ 45.762389] do_syscall_64+0x55/0xf0 > [ 45.783393] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > [ 45.806505] RIP: 0033:0x7f59cd765686 > [ 45.828537] Code: 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 > 00 00 48 63 d7 be 69 19 12 28 48 c7 c7 ad de e1 fe b8 a9 00 00 00 0f 05 > <48> > 3d 00 f0 ff ff 77 02 f3 c3 48 8b 15 e1 77 2b 00 f7 d8 64 89 02 > [ 45.886146] RSP: 002b:00007ffe18fd6c88 EFLAGS: 00000206 ORIG_RAX: > 00000000000000a9 > [ 45.914428] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: > 00007f59cd765686 > [ 45.943101] RDX: 000000004321fedc RSI: 0000000028121969 RDI: > fffffffffee1dead > [ 45.971078] RBP: 0000000000000000 R08: 0000000000000100 R09: > 0000000000000005 > [ 46.000322] R10: 000000000000003d R11: 0000000000000206 R12: > 00007ffe18fd6d20 > [ 46.029825] R13: 00005629faff6360 R14: 00007ffe18fd6dc0 R15: > 00007ffe18fd7018 > [ 46.059573] Modules linked in: nls_ascii(E) nls_cp437(E) vfat(E) fat(E) > intel_rapl(E) sb_edac(E) x86_pkg_temp_thermal(E) intel_powerclamp(E) > coretemp(E) kvm_intel(E) kvm(E) irqbypass(E) crct10dif_pclmul(E) > crc32_pclmul(E) ghash_clmulni_intel(E) aesni_intel(E) aes_x86_64(E) > crypto_simd(E) cryptd(E) glue_helper(E) intel_cstate(E) iTCO_wdt(E) > intel_uncore(E) iTCO_vendor_support(E) snd_pcm(E) efi_pstore(E) > intel_rapl_perf(E) snd_timer(E) snd(E) soundcore(E) efivars(E) pcspkr(E) > lpc_ich(E) joydev(E) evdev(E) bfq(E) mei_me(E) sg(E) mei(E) ioatdma(E) > ipmi_si(E) ipmi_devintf(E) ipmi_msghandler(E) pcc_cpufreq(E) button(E) > efivarfs(E) ip_tables(E) x_tables(E) autofs4(E) ext4(E) crc16(E) > mbcache(E) > jbd2(E) fscrypto(E) btrfs(E) zstd_decompress(E) zstd_compress(E) xxhash(E) > raid0(E) multipath(E) raid456(E) async_raid6_recov(E) async_memcpy(E) > async_pq(E) async_xor(E) async_tx(E) xor(E) raid1(E) raid6_pq(E) > libcrc32c(E) crc32c_generic(E) raid10(E) linear(E) ses(E) md_mod(E) > enclosure(E) hid_generic(E) > [ 46.059627] usbhid(E) hid(E) sd_mod(E) ahci(E) libahci(E) ehci_pci(E) > mpt3sas(E) raid_class(E) igb(E) ehci_hcd(E) scsi_transport_sas(E) > libata(E) > i2c_algo_bit(E) crc32c_intel(E) usbcore(E) scsi_mod(E) i2c_i801(E) dca(E) > wmi(E) > [ 46.453696] ---[ end trace c7f8fcdd896d33e6 ]--- > [ 46.496353] RIP: 0010:_scsih_set_satl_pending+0xc/0x50 [mpt3sas] > [ 46.536771] Code: c3 48 c1 ee 0b b8 20 00 00 00 ba 40 00 00 00 89 41 04 > 89 11 31 c0 89 71 08 c3 0f 1f 40 00 0f 1f 44 00 00 48 8b 87 f8 00 00 00 > <0f> > b6 10 80 fa a1 74 09 31 c0 80 fa 85 74 02 f3 c3 48 8b 47 38 40 > [ 46.626011] RSP: 0018:ffffbb31000bfd58 EFLAGS: 00010202 > [ 46.665499] RAX: 5d8824840b490000 RBX: ffff9179613c47b8 RCX: > 0000000000000000 > [ 46.706433] RDX: ffff9179613c4680 RSI: 0000000000000000 RDI: > ffff9179613c47b8 > [ 46.747730] RBP: 0000000000000d48 R08: 0000000000000002 R09: > ffffbb31000bfd24 > [ 46.788281] R10: 0000000000000000 R11: 0000000000002eaf R12: > ffff91796d4d67a8 > [ 46.829094] R13: 0000000000000001 R14: ffff91796cd87110 R15: > 0000000000000000 > [ 46.873644] FS: 00007f59cdc31900(0000) GS:ffff917d6fa00000(0000) > knlGS:0000000000000000 > [ 46.919720] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 46.958743] CR2: 00007f3caf6258dc CR3: 000000046b4f0006 CR4: > 00000000001606e0 > [ 47.002844] printk: systemd-shutdow: 4 output lines suppressed due to > ratelimiting > [ 47.044818] Kernel panic - not syncing: Attempted to kill init! > exitcode=0x0000000b > [ 47.086916] Kernel Offset: 0x36600000 from 0xffffffff81000000 > (relocation > range: 0xffffffff80000000-0xffffffffbfffffff) > [ 47.136709] ---[ end Kernel panic - not syncing: Attempted to kill > init! > exitcode=0x0000000b ]--- > [ 47.180672] ------------[ cut here ]------------ > [ 47.220188] sched: Unexpected reschedule of offline CPU#0! > [ 47.260447] WARNING: CPU: 5 PID: 1 at arch/x86/kernel/smp.c:128 > native_smp_send_reschedule+0x34/0x40 > [ 47.305700] Modules linked in: nls_ascii(E) nls_cp437(E) vfat(E) fat(E) > intel_rapl(E) sb_edac(E) x86_pkg_temp_thermal(E) intel_powerclamp(E) > coretemp(E) kvm_intel(E) kvm(E) irqbypass(E) crct10dif_pclmul(E) > crc32_pclmul(E) ghash_clmulni_intel(E) aesni_intel(E) aes_x86_64(E) > crypto_simd(E) cryptd(E) glue_helper(E) intel_cstate(E) iTCO_wdt(E) > intel_uncore(E) iTCO_vendor_support(E) snd_pcm(E) efi_pstore(E) > intel_rapl_perf(E) snd_timer(E) snd(E) soundcore(E) efivars(E) pcspkr(E) > lpc_ich(E) joydev(E) evdev(E) bfq(E) mei_me(E) sg(E) mei(E) ioatdma(E) > ipmi_si(E) ipmi_devintf(E) ipmi_msghandler(E) pcc_cpufreq(E) button(E) > efivarfs(E) ip_tables(E) x_tables(E) autofs4(E) ext4(E) crc16(E) > mbcache(E) > jbd2(E) fscrypto(E) btrfs(E) zstd_decompress(E) zstd_compress(E) xxhash(E) > raid0(E) multipath(E) raid456(E) async_raid6_recov(E) async_memcpy(E) > async_pq(E) async_xor(E) async_tx(E) xor(E) raid1(E) raid6_pq(E) > libcrc32c(E) crc32c_generic(E) raid10(E) linear(E) ses(E) md_mod(E) > enclosure(E) hid_generic(E) > [ 47.305718] usbhid(E) hid(E) sd_mod(E) ahci(E) libahci(E) ehci_pci(E) > mpt3sas(E) raid_class(E) igb(E) ehci_hcd(E) scsi_transport_sas(E) > libata(E) > i2c_algo_bit(E) crc32c_intel(E) usbcore(E) scsi_mod(E) i2c_i801(E) dca(E) > wmi(E) > [ 47.826473] CPU: 5 PID: 1 Comm: systemd-shutdow Tainted: G D E > 4.20.7 #2 > [ 47.878173] Hardware name: Supermicro X9DR7/E-(J)LN4F/X9DR7/E-(J)LN4F, > BIOS 3.3 07/13/2018 > [ 47.930647] RIP: 0010:native_smp_send_reschedule+0x34/0x40 > [ 47.979938] Code: 05 f1 f8 0b 01 73 15 48 8b 05 e8 81 e7 00 be fd 00 00 > 00 48 8b 40 30 e9 ea 56 9b 00 89 fe 48 c7 c7 48 51 42 b8 e8 ac 20 03 00 > <0f> > 0b c3 66 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 53 48 83 ec 20 > [ 48.093029] RSP: 0018:ffff917d6fa83ed8 EFLAGS: 00010082 > [ 48.149498] RAX: 0000000000000000 RBX: ffff917d6ddf63c0 RCX: > ffffffffb864d428 > [ 48.202396] RDX: 0000000000000001 RSI: 0000000000000092 RDI: > 0000000000000046 > [ 48.254816] RBP: 0000000000000000 R08: 0000000000000005 R09: > 0000000000aaaaaa > [ 48.306828] R10: 0000000000000020 R11: 0000000000000001 R12: > ffffbb31000bfda8 > [ 48.357835] R13: ffffffffb7710cd0 R14: 0000000000000006 R15: > 0000000000000000 > [ 48.407961] FS: 0000000000000000(0000) GS:ffff917d6fa80000(0000) > knlGS:0000000000000000 > [ 48.458603] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 48.506629] CR2: 00007fb6b8010938 CR3: 000000067f00a004 CR4: > 00000000001606e0 > [ 48.556242] Call Trace: > [ 48.598783] <IRQ> > [ 48.639859] update_process_times+0x40/0x50 > [ 48.682328] tick_sched_handle+0x22/0x60 > [ 48.723880] tick_sched_timer+0x37/0x70 > [ 48.765024] __hrtimer_run_queues+0xfc/0x270 > [ 48.805900] hrtimer_interrupt+0x101/0x240 > [ 48.846134] smp_apic_timer_interrupt+0x6a/0x130 > [ 48.886532] apic_timer_interrupt+0xf/0x20 > [ 48.925831] </IRQ> > [ 48.962622] RIP: 0010:panic+0x21b/0x262 > [ 49.001520] Code: eb a6 83 3d 58 ac 53 01 00 74 05 e8 f1 59 02 00 48 c7 > c6 60 90 bb b8 48 c7 c7 a0 ee 42 b8 e8 dc 37 06 00 fb 66 0f 1f 44 00 00 > <31> > db e8 60 72 0c 00 4c 39 eb 7c 1d 41 83 f4 01 48 8b 05 00 ac 53 > [ 49.093776] RSP: 0018:ffffbb31000bfe58 EFLAGS: 00000282 ORIG_RAX: > ffffffffffffff13 > [ 49.138456] RAX: 0000000000000055 RBX: ffff917d6ddf6300 RCX: > ffffffffb864d428 > [ 49.181707] RDX: 0000000000000000 RSI: 0000000000000096 RDI: > 0000000000000046 > [ 49.225590] RBP: ffffbb31000bfed0 R08: 0000000000000005 R09: > 0000000000aaaaaa > [ 49.268823] R10: 0000000000000019 R11: 0000000000000001 R12: > 0000000000000000 > [ 49.311527] R13: 0000000000000000 R14: ffffffffb86452e0 R15: > ffff917d6ddf63c0 > [ 49.357866] ? panic+0x214/0x262 > [ 49.395102] do_exit+0xc0c/0xc10 > [ 49.432198] ? do_writev+0x5e/0xf0 > [ 49.468598] rewind_stack_do_exit+0x17/0x20 > [ 49.505879] ---[ end trace c7f8fcdd896d33e7 ]--- > [ 49.543704] WARNING: CPU: 5 PID: 1 at kernel/sched/core.c:1168 > set_task_cpu+0x167/0x170 > [ 49.588858] Modules linked in: nls_ascii(E) nls_cp437(E) vfat(E) fat(E) > intel_rapl(E) sb_edac(E) x86_pkg_temp_thermal(E) intel_powerclamp(E) > coretemp(E) kvm_intel(E) kvm(E) irqbypass(E) crct10dif_pclmul(E) > crc32_pclmul(E) ghash_clmulni_intel(E) aesni_intel(E) aes_x86_64(E) > crypto_simd(E) cryptd(E) glue_helper(E) intel_cstate(E) iTCO_wdt(E) > intel_uncore(E) iTCO_vendor_support(E) snd_pcm(E) efi_pstore(E) > intel_rapl_perf(E) snd_timer(E) snd(E) soundcore(E) efivars(E) pcspkr(E) > lpc_ich(E) joydev(E) evdev(E) bfq(E) mei_me(E) sg(E) mei(E) ioatdma(E) > ipmi_si(E) ipmi_devintf(E) ipmi_msghandler(E) pcc_cpufreq(E) button(E) > efivarfs(E) ip_tables(E) x_tables(E) autofs4(E) ext4(E) crc16(E) > mbcache(E) > jbd2(E) fscrypto(E) btrfs(E) zstd_decompress(E) zstd_compress(E) xxhash(E) > raid0(E) multipath(E) raid456(E) async_raid6_recov(E) async_memcpy(E) > async_pq(E) async_xor(E) async_tx(E) xor(E) raid1(E) raid6_pq(E) > libcrc32c(E) crc32c_generic(E) raid10(E) linear(E) ses(E) md_mod(E) > enclosure(E) hid_generic(E) > [ 49.588874] usbhid(E) hid(E) sd_mod(E) ahci(E) libahci(E) ehci_pci(E) > mpt3sas(E) raid_class(E) igb(E) ehci_hcd(E) scsi_transport_sas(E) > libata(E) > i2c_algo_bit(E) crc32c_intel(E) usbcore(E) scsi_mod(E) i2c_i801(E) dca(E) > wmi(E) > [ 50.036919] CPU: 5 PID: 1 Comm: systemd-shutdow Tainted: G D W E > 4.20.7 #2 > [ 50.078246] Hardware name: Supermicro X9DR7/E-(J)LN4F/X9DR7/E-(J)LN4F, > BIOS 3.3 07/13/2018 > [ 50.118827] RIP: 0010:set_task_cpu+0x167/0x170 > [ 50.155031] Code: ff ff ff 80 8b 4c 04 00 00 04 e9 54 ff ff ff 0f 0b e9 > de fe ff ff f7 43 60 fd ff ff ff 0f 84 e8 fe ff ff 0f 0b e9 e1 fe ff ff > <0f> > 0b e9 eb fe ff ff 66 90 0f 1f 44 00 00 41 55 41 54 49 89 f5 55 > [ 50.240066] RSP: 0018:ffff917d6fa83e58 EFLAGS: 00010006 > [ 50.278297] RAX: 0000000000000200 RBX: ffff91796d40e3c0 RCX: > 000000000000018a > [ 50.319641] RDX: ffffffffffffff58 RSI: 0000000000000006 RDI: > ffff91796d40e3c0 > [ 50.360526] RBP: ffff91796d40e728 R08: 0000000000000006 R09: > 0000000000021240 > [ 50.401357] R10: 0000000000000005 R11: 0000000000000001 R12: > 0000000000000006 > [ 50.442145] R13: 0000000000000006 R14: 0000000000000046 R15: > 0000000000021980 > [ 50.482260] FS: 0000000000000000(0000) GS:ffff917d6fa80000(0000) > knlGS:0000000000000000 > [ 50.524269] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 50.564296] CR2: 00007fb6b8010938 CR3: 000000067f00a004 CR4: > 00000000001606e0 > [ 50.605120] Call Trace: > [ 50.641247] <IRQ> > [ 50.676445] try_to_wake_up+0x151/0x490 > [ 50.713369] swake_up_locked+0x20/0x50 > [ 50.750958] swake_up_one+0x1a/0x30 > [ 50.787796] rcu_process_callbacks+0xb3/0x460 > [ 50.826035] __do_softirq+0x112/0x2e0 > [ 50.867060] irq_exit+0xc2/0xd0 > [ 50.908429] smp_apic_timer_interrupt+0x74/0x130 > [ 50.946337] apic_timer_interrupt+0xf/0x20 > [ 50.983312] </IRQ> > [ 51.016891] RIP: 0010:panic+0x21b/0x262 > [ 51.053172] Code: eb a6 83 3d 58 ac 53 01 00 74 05 e8 f1 59 02 00 48 c7 > c6 60 90 bb b8 48 c7 c7 a0 ee 42 b8 e8 dc 37 06 00 fb 66 0f 1f 44 00 00 > <31> > db e8 60 72 0c 00 4c 39 eb 7c 1d 41 83 f4 01 48 8b 05 00 ac 53 > [ 51.138104] RSP: 0018:ffffbb31000bfe58 EFLAGS: 00000282 ORIG_RAX: > ffffffffffffff13 > [ 51.178857] RAX: 0000000000000055 RBX: ffff917d6ddf6300 RCX: > ffffffffb864d428 > [ 51.219368] RDX: 0000000000000000 RSI: 0000000000000096 RDI: > 0000000000000046 > [ 51.259723] RBP: ffffbb31000bfed0 R08: 0000000000000005 R09: > 0000000000aaaaaa > [ 51.299906] R10: 0000000000000019 R11: 0000000000000001 R12: > 0000000000000000 > [ 51.340981] R13: 0000000000000000 R14: ffffffffb86452e0 R15: > ffff917d6ddf63c0 > [ 51.381547] ? panic+0x214/0x262 > [ 51.417817] do_exit+0xc0c/0xc10 > [ 51.454187] ? do_writev+0x5e/0xf0 > [ 51.490356] rewind_stack_do_exit+0x17/0x20 > [ 51.526915] ---[ end trace c7f8fcdd896d33e8 ]--- > [ 51.563684] ------------[ cut here ]------------ > [ 51.598307] sched: Unexpected reschedule of offline CPU#6! > [ 51.634384] WARNING: CPU: 5 PID: 1 at arch/x86/kernel/smp.c:128 > native_smp_send_reschedule+0x34/0x40 > [ 51.673548] Modules linked in: nls_ascii(E) nls_cp437(E) vfat(E) fat(E) > intel_rapl(E) sb_edac(E) x86_pkg_temp_thermal(E) intel_powerclamp(E) > coretemp(E) kvm_intel(E) kvm(E) irqbypass(E) crct10dif_pclmul(E) > crc32_pclmul(E) ghash_clmulni_intel(E) aesni_intel(E) aes_x86_64(E) > crypto_simd(E) cryptd(E) glue_helper(E) intel_cstate(E) iTCO_wdt(E) > intel_uncore(E) iTCO_vendor_support(E) snd_pcm(E) efi_pstore(E) > intel_rapl_perf(E) snd_timer(E) snd(E) soundcore(E) efivars(E) pcspkr(E) > lpc_ich(E) joydev(E) evdev(E) bfq(E) mei_me(E) sg(E) mei(E) ioatdma(E) > ipmi_si(E) ipmi_devintf(E) ipmi_msghandler(E) pcc_cpufreq(E) button(E) > efivarfs(E) ip_tables(E) x_tables(E) autofs4(E) ext4(E) crc16(E) > mbcache(E) > jbd2(E) fscrypto(E) btrfs(E) zstd_decompress(E) zstd_compress(E) xxhash(E) > raid0(E) multipath(E) raid456(E) async_raid6_recov(E) async_memcpy(E) > async_pq(E) async_xor(E) async_tx(E) xor(E) raid1(E) raid6_pq(E) > libcrc32c(E) crc32c_generic(E) raid10(E) linear(E) ses(E) md_mod(E) > enclosure(E) hid_generic(E) > [ 51.673564] usbhid(E) hid(E) sd_mod(E) ahci(E) libahci(E) ehci_pci(E) > mpt3sas(E) raid_class(E) igb(E) ehci_hcd(E) scsi_transport_sas(E) > libata(E) > i2c_algo_bit(E) crc32c_intel(E) usbcore(E) scsi_mod(E) i2c_i801(E) dca(E) > wmi(E) > [ 52.089150] CPU: 5 PID: 1 Comm: systemd-shutdow Tainted: G D W E > 4.20.7 #2 > [ 52.132938] Hardware name: Supermicro X9DR7/E-(J)LN4F/X9DR7/E-(J)LN4F, > BIOS 3.3 07/13/2018 > [ 52.173615] RIP: 0010:native_smp_send_reschedule+0x34/0x40 > [ 52.210372] Code: 05 f1 f8 0b 01 73 15 48 8b 05 e8 81 e7 00 be fd 00 00 > 00 48 8b 40 30 e9 ea 56 9b 00 89 fe 48 c7 c7 48 51 42 b8 e8 ac 20 03 00 > <0f> > 0b c3 66 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 53 48 83 ec 20 > [ 52.294550] RSP: 0018:ffff917d6fa83e40 EFLAGS: 00010086 > [ 52.333796] RAX: 0000000000000000 RBX: ffff917d6fb21980 RCX: > ffffffffb864d428 > [ 52.374430] RDX: 0000000000000001 RSI: 0000000000000086 RDI: > 0000000000000046 > [ 52.414733] RBP: ffff91796d40e3c0 R08: 0000000000000005 R09: > 0000000000aaaaaa > [ 52.455089] R10: ffff917d6fa83d78 R11: 0000000000000001 R12: > ffff917d6fb21980 > [ 52.495131] R13: ffff917d6fa83e90 R14: 0000000000000046 R15: > 0000000000021980 > [ 52.535792] FS: 0000000000000000(0000) GS:ffff917d6fa80000(0000) > knlGS:0000000000000000 > [ 52.577984] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 52.617169] CR2: 00007fb6b8010938 CR3: 000000067f00a004 CR4: > 00000000001606e0 > [ 52.657878] Call Trace: > [ 52.694014] <IRQ> > [ 52.729620] check_preempt_curr+0x4e/0x90 > [ 52.767410] ttwu_do_wakeup+0x19/0x140 > [ 52.804758] try_to_wake_up+0x1ce/0x490 > [ 52.842279] swake_up_locked+0x20/0x50 > [ 52.879522] swake_up_one+0x1a/0x30 > [ 52.916160] rcu_process_callbacks+0xb3/0x460 > [ 52.953326] __do_softirq+0x112/0x2e0 > [ 52.989287] irq_exit+0xc2/0xd0 > [ 53.023912] smp_apic_timer_interrupt+0x74/0x130 > [ 53.061088] apic_timer_interrupt+0xf/0x20 > [ 53.096701] </IRQ> > [ 53.130400] RIP: 0010:panic+0x21b/0x262 > [ 53.165971] Code: eb a6 83 3d 58 ac 53 01 00 74 05 e8 f1 59 02 00 48 c7 > c6 60 90 bb b8 48 c7 c7 a0 ee 42 b8 e8 dc 37 06 00 fb 66 0f 1f 44 00 00 > <31> > db e8 60 72 0c 00 4c 39 eb 7c 1d 41 83 f4 01 48 8b 05 00 ac 53 > [ 53.250400] RSP: 0018:ffffbb31000bfe58 EFLAGS: 00000282 ORIG_RAX: > ffffffffffffff13 > [ 53.291294] RAX: 0000000000000055 RBX: ffff917d6ddf6300 RCX: > ffffffffb864d428 > [ 53.334309] RDX: 0000000000000000 RSI: 0000000000000096 RDI: > 0000000000000046 > [ 53.376528] RBP: ffffbb31000bfed0 R08: 0000000000000005 R09: > 0000000000aaaaaa > [ 53.417038] R10: 0000000000000019 R11: 0000000000000001 R12: > 0000000000000000 > [ 53.457751] R13: 0000000000000000 R14: ffffffffb86452e0 R15: > ffff917d6ddf63c0 > [ 53.498695] ? panic+0x214/0x262 > [ 53.535368] do_exit+0xc0c/0xc10 > [ 53.574925] ? do_writev+0x5e/0xf0 > [ 53.614015] rewind_stack_do_exit+0x17/0x20 > [ 53.653035] ---[ end trace c7f8fcdd896d33e9 ]--- >