Hi, I’m seeing numerous crashes on the xen 4.6.6-1 / 4.6.6-2 releases, on both the 4.9.34-29 and 4.9.39-29 kernels. I’ve attached a txt with two different servers outputs. Xen-028: This crashed this morning while running 4.6.6-1 and 4.9.39-29 Xen-001: This crashed shortly after being upgraded to 4.6.6-2 and 4.9.34-29 Both are on different hardware platforms, and have had a long history of being stable until these upgrades. It sounds potentially related to https://kernel.googlesource.com/pub/scm/linux/kernel/git/tiwai/sound-unstable/+/9ce119f318ba1a07c29149301f1544b6c4bea52a%5E%21/ but I’ve confirmed this patch is in the above kernels. Any suggestions / thoughts? Cheers, Nathan |
Aug 23 10:19:31 xen-028 kernel: [590071.735515] BUG: unable to handle kernel paging request at 0000000000002260 Aug 23 10:19:31 xen-028 kernel: [590071.735795] IP: [<ffffffff8152e6a4>] n_tty_receive_buf_common+0xa4/0x1f0 Aug 23 10:19:31 xen-028 kernel: [590071.736031] PGD 0 Aug 23 10:19:31 xen-028 kernel: [590071.736083] Aug 23 10:19:31 xen-028 kernel: [590071.736300] Oops: 0000 [#1] SMP Aug 23 10:19:31 xen-028 kernel: [590071.736470] Modules linked in: ebt_ip6 ebt_ip ebtable_filter ebtables arptable_filter arp_tables bridge xen_pciback xen_gntalloc nfsd auth_rpcgss nfsv3 nfs_acl nfs fscache lockd sunrpc grace 8021q mrp garp stp llc bonding blktap xen_netback xen_blkback xen_gntdev xen_evtchn xenfs xen_privcmd ipmi_devintf ipmi_si ipmi_msghandler gpio_ich iTCO_wdt iTCO_vendor_support fjes acpi_power_meter dcdbas pcspkr serio_raw joydev lpc_ich igb ixgbe dca ptp pps_core mdio i7core_edac edac_core bnx2 raid1 megaraid_sas ttm Aug 23 10:19:31 xen-028 kernel: [590071.740051] CPU: 14 PID: 21615 Comm: kworker/u48:1 Not tainted 4.9.39-29.el6.x86_64 #1 Aug 23 10:19:31 xen-028 kernel: [590071.740330] Hardware name: Dell Inc. PowerEdge R610/0F0XJ6, BIOS 6.0.7 08/18/2011 Aug 23 10:19:31 xen-028 kernel: [590071.740607] Workqueue: events_unbound flush_to_ldisc Aug 23 10:19:31 xen-028 kernel: [590071.740806] task: ffff88008a6011c0 task.stack: ffffc9004cfec000 Aug 23 10:19:31 xen-028 kernel: [590071.740966] RIP: e030:[<ffffffff8152e6a4>] [<ffffffff8152e6a4>] n_tty_receive_buf_common+0xa4/0x1f0 Aug 23 10:19:31 xen-028 kernel: [590071.741282] RSP: e02b:ffffc9004cfefb08 EFLAGS: 00010296 Aug 23 10:19:31 xen-028 kernel: [590071.741442] RAX: 0000000000002260 RBX: 0000000000000000 RCX: 000000000000000a Aug 23 10:19:31 xen-028 kernel: [590071.741714] RDX: 0000000000000000 RSI: ffff88015ecd6420 RDI: ffff8800afd654d8 Aug 23 10:19:31 xen-028 kernel: [590071.741994] RBP: ffffc9004cfefb78 R08: 0000000000000001 R09: ffffffff81f0af00 Aug 23 10:19:31 xen-028 kernel: [590071.742274] R10: 0000000000007ff0 R11: 0000000000000078 R12: 000000000000000a Aug 23 10:19:31 xen-028 kernel: [590071.742549] R13: ffff8800afd65400 R14: 0000000000000000 R15: ffff88015ecd6420 Aug 23 10:19:31 xen-028 kernel: [590071.742830] FS: 00007f81da7317c0(0000) GS:ffff8801c0980000(0000) knlGS:0000000000000000 Aug 23 10:19:31 xen-028 kernel: [590071.743112] CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 23 10:19:31 xen-028 kernel: [590071.743283] CR2: 0000000000002260 CR3: 000000008f61f000 CR4: 0000000000002660 Aug 23 10:19:31 xen-028 kernel: [590071.743564] Stack: Aug 23 10:19:31 xen-028 kernel: [590071.743719] ffffc9001160000c 0000000000000000 ffff8800afd654d8 00000001c0999970 Aug 23 10:19:31 xen-028 kernel: [590071.744149] 0000000000002260 000000008a603340 ffff8801c0997000 0000000000000000 Aug 23 10:19:31 xen-028 kernel: [590071.744577] ffff8801c098b890 ffff88015ecd6400 ffff8800b19e9c00 ffffc9004cfefbf8 Aug 23 10:19:31 xen-028 kernel: [590071.745008] Call Trace: Aug 23 10:19:31 xen-028 kernel: [590071.745169] [<ffffffff8152e804>] n_tty_receive_buf2+0x14/0x20 Aug 23 10:19:31 xen-028 kernel: [590071.745335] [<ffffffff81531533>] tty_ldisc_receive_buf+0x23/0x50 Aug 23 10:19:31 xen-028 kernel: [590071.745501] [<ffffffff81531958>] flush_to_ldisc+0xc8/0x100 Aug 23 10:19:31 xen-028 kernel: [590071.745669] [<ffffffff8102eb3c>] ? __switch_to+0x1dc/0x680 Aug 23 10:19:31 xen-028 kernel: [590071.745836] [<ffffffff810c0490>] process_one_work+0x170/0x500 Aug 23 10:19:31 xen-028 kernel: [590071.746005] [<ffffffff818d4658>] ? __schedule+0x238/0x530 Aug 23 10:19:31 xen-028 kernel: [590071.746169] [<ffffffff810c1234>] ? maybe_create_worker+0x94/0x120 Aug 23 10:19:31 xen-028 kernel: [590071.746342] [<ffffffff818d4a3a>] ? schedule+0x3a/0xa0 Aug 23 10:19:31 xen-028 kernel: [590071.746506] [<ffffffff810c1426>] worker_thread+0x166/0x580 Aug 23 10:19:31 xen-028 kernel: [590071.746671] [<ffffffff818d4658>] ? __schedule+0x238/0x530 Aug 23 10:19:31 xen-028 kernel: [590071.749537] [<ffffffff810d3882>] ? default_wake_function+0x12/0x20 Aug 23 10:19:31 xen-028 kernel: [590071.749706] [<ffffffff810c12c0>] ? maybe_create_worker+0x120/0x120 Aug 23 10:19:31 xen-028 kernel: [590071.749872] [<ffffffff818d4a3a>] ? schedule+0x3a/0xa0 Aug 23 10:19:31 xen-028 kernel: [590071.750040] [<ffffffff818d8826>] ? _raw_spin_unlock_irqrestore+0x16/0x20 Aug 23 10:19:31 xen-028 kernel: [590071.750204] [<ffffffff810c12c0>] ? maybe_create_worker+0x120/0x120 Aug 23 10:19:31 xen-028 kernel: [590071.750369] [<ffffffff810c62c5>] kthread+0xe5/0x100 Aug 23 10:19:31 xen-028 kernel: [590071.750532] [<ffffffff810c61e0>] ? __kthread_init_worker+0x40/0x40 Aug 23 10:19:31 xen-028 kernel: [590071.750698] [<ffffffff818d8f55>] ret_from_fork+0x25/0x30 Aug 23 10:19:31 xen-028 kernel: [590071.750860] Code: 89 fe 4c 89 ef 89 45 98 e8 aa fb ff ff 8b 45 98 48 63 d0 48 85 db 48 8d 0c 13 48 0f 45 d9 01 45 bc 49 01 d7 41 29 c4 48 8b 45 b0 <48> 8b 30 48 89 75 c0 49 8b 0e 8d 96 00 10 00 00 29 ca 41 f6 85 Aug 23 10:19:31 xen-028 kernel: [590071.753725] RIP [<ffffffff8152e6a4>] n_tty_receive_buf_common+0xa4/0x1f0 Aug 23 10:19:31 xen-028 kernel: [590071.753928] RSP <ffffc9004cfefb08> Aug 23 10:19:31 xen-028 kernel: [590071.754087] CR2: 0000000000002260 Aug 23 10:19:31 xen-028 kernel: [590071.754247] ---[ end trace 3533c918d837d330 ]--- Aug 23 10:19:31 xen-028 kernel: [590071.760173] BUG: unable to handle kernel paging request at ffffffffffffffd8 Aug 23 10:19:31 xen-028 kernel: [590071.760422] IP: [<ffffffff810c5aa0>] kthread_data+0x10/0x20 Aug 23 10:19:31 xen-028 kernel: [590071.760632] PGD 1e0a067 Aug 23 10:19:31 xen-028 kernel: [590071.760676] PUD 1e0c067 Aug 23 10:19:31 xen-028 kernel: [590071.760871] PMD 0 Aug 23 10:19:31 xen-028 kernel: [590071.760910] Aug 23 10:19:31 xen-028 kernel: [590071.761103] Oops: 0000 [#2] SMP Aug 23 10:19:31 xen-028 kernel: [590071.761262] Modules linked in: ebt_ip6 ebt_ip ebtable_filter ebtables arptable_filter arp_tables bridge xen_pciback xen_gntalloc nfsd auth_rpcgss nfsv3 nfs_acl nfs fscache lockd sunrpc grace 8021q mrp garp stp llc bonding blktap xen_netback xen_blkback xen_gntdev xen_evtchn xenfs xen_privcmd ipmi_devintf ipmi_si ipmi_msghandler gpio_ich iTCO_wdt iTCO_vendor_support fjes acpi_power_meter dcdbas pcspkr serio_raw joydev lpc_ich igb ixgbe dca ptp pps_core mdio i7core_edac edac_core bnx2 raid1 megaraid_sas ttm Aug 23 10:19:31 xen-028 kernel: [590071.764349] CPU: 14 PID: 21615 Comm: kworker/u48:1 Tainted: G D 4.9.39-29.el6.x86_64 #1 Aug 23 10:19:31 xen-028 kernel: [590071.764633] Hardware name: Dell Inc. PowerEdge R610/0F0XJ6, BIOS 6.0.7 08/18/2011 Aug 23 10:19:31 xen-028 kernel: [590071.764928] task: ffff88008a6011c0 task.stack: ffffc9004cfec000 Aug 23 10:19:31 xen-028 kernel: [590071.765092] RIP: e030:[<ffffffff810c5aa0>] [<ffffffff810c5aa0>] kthread_data+0x10/0x20 Aug 23 10:19:31 xen-028 kernel: [590071.765412] RSP: e02b:ffffc9004cfefdd8 EFLAGS: 00010086 Aug 23 10:19:31 xen-028 kernel: [590071.765579] RAX: 0000000000000000 RBX: ffff8801c0999900 RCX: 000000000000000e Aug 23 10:19:31 xen-028 kernel: [590071.765858] RDX: ffff8801bc009400 RSI: ffff88008a6011c0 RDI: ffff88008a6011c0 Aug 23 10:19:31 xen-028 kernel: [590071.766135] RBP: ffffc9004cfefdd8 R08: ffff8801c0980000 R09: 0000000000000001 Aug 23 10:19:31 xen-028 kernel: [590071.766415] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000019900 Aug 23 10:19:31 xen-028 kernel: [590071.766693] R13: ffff88008a6011c0 R14: 0000000000000000 R15: ffff88008a601b80 Aug 23 10:19:31 xen-028 kernel: [590071.766982] FS: 00007f81da7317c0(0000) GS:ffff8801c0980000(0000) knlGS:0000000000000000 Aug 23 10:19:31 xen-028 kernel: [590071.767276] CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 23 10:19:31 xen-028 kernel: [590071.767441] CR2: 0000000000000028 CR3: 000000008f61f000 CR4: 0000000000002660 Aug 23 10:19:31 xen-028 kernel: [590071.767727] Stack: Aug 23 10:19:31 xen-028 kernel: [590071.767886] ffffc9004cfefe08 ffffffff810bd282 ffffc9004cfefdf8 ffff8801c0999900 Aug 23 10:19:31 xen-028 kernel: [590071.768332] 0000000000019900 ffff88008a6011c0 ffffc9004cfefe78 ffffffff818d4834 Aug 23 10:19:31 xen-028 kernel: [590071.768762] 0000000000000001 ffff8801aad54000 ffffc9004cfefe48 ffff8801ab7b5c08 Aug 23 10:19:31 xen-028 kernel: [590071.769185] Call Trace: Aug 23 10:19:31 xen-028 kernel: [590071.769343] [<ffffffff810bd282>] wq_worker_sleeping+0x12/0xa0 Aug 23 10:19:31 xen-028 kernel: [590071.769506] [<ffffffff818d4834>] __schedule+0x414/0x530 Aug 23 14:29:55 xen-001 kernel: [15199.824132] BUG: unable to handle kernel paging request at 0000000000002260 Aug 23 14:29:55 xen-001 kernel: [15199.824541] IP: [<ffffffff8152e6a4>] n_tty_receive_buf_common+0xa4/0x1f0 Aug 23 14:29:55 xen-001 kernel: [15199.824885] PGD 0 Aug 23 14:29:55 xen-001 kernel: [15199.824950] Aug 23 14:29:55 xen-001 kernel: [15199.825274] Oops: 0000 [#1] SMP Aug 23 14:29:55 xen-001 kernel: [15199.825541] Modules linked in: mpt3sas scsi_transport_sas raid_class mptctl mptbase dell_rbu ebt_ip6 ebt_ip ebtable_filter ebtables arptable_filter arp_tables bridge xen_pciback xen_gntalloc nfsd $ Aug 23 14:29:55 xen-001 kernel: [15199.830906] CPU: 2 PID: 11441 Comm: kworker/u48:2 Not tainted 4.9.39-29.el6.x86_64 #1 Aug 23 14:29:55 xen-001 kernel: [15199.831383] Hardware name: Dell Inc. PowerEdge C6220/03C9JJ, BIOS 1.1.19 02/25/2013 Aug 23 14:29:55 xen-001 kernel: [15199.831867] Workqueue: events_unbound flush_to_ldisc Aug 23 14:29:55 xen-001 kernel: [15199.832197] task: ffff88004f5cd240 task.stack: ffffc90060ebc000 Aug 23 14:29:55 xen-001 kernel: [15199.832470] RIP: e030:[<ffffffff8152e6a4>] [<ffffffff8152e6a4>] n_tty_receive_buf_common+0xa4/0x1f0 Aug 23 14:29:55 xen-001 kernel: [15199.833011] RSP: e02b:ffffc90060ebfb08 EFLAGS: 00010296 Aug 23 14:29:55 xen-001 kernel: [15199.833281] RAX: 0000000000002260 RBX: 0000000000000000 RCX: 0000000000000004 Aug 23 14:29:55 xen-001 kernel: [15199.833556] RDX: 0000000000000000 RSI: ffff88006f9ef020 RDI: ffff8801c8c5f0d8 Aug 23 14:29:55 xen-001 kernel: [15199.833830] RBP: ffffc90060ebfb78 R08: 0000000000000001 R09: ffffffff81f0af00 Aug 23 14:29:55 xen-001 kernel: [15199.834105] R10: 0000000000007ff0 R11: 0000000000000078 R12: 0000000000000004 Aug 23 14:29:55 xen-001 kernel: [15199.834378] R13: ffff8801c8c5f000 R14: 0000000000000000 R15: ffff88006f9ef020 Aug 23 14:29:55 xen-001 kernel: [15199.834657] FS: 00007f65711087c0(0000) GS:ffff880201a80000(0000) knlGS:0000000000000000 Aug 23 14:29:55 xen-001 kernel: [15199.835131] CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033 Aug 23 14:29:55 xen-001 kernel: [15199.835402] CR2: 0000000000002260 CR3: 00000001f6dbe000 CR4: 0000000000042660 Aug 23 14:29:55 xen-001 kernel: [15199.835676] Stack: Aug 23 14:29:55 xen-001 kernel: [15199.835938] ffffc90060ebfb38 0000000000000000 ffff8801c8c5f0d8 0000000101a99970 Aug 23 14:29:55 xen-001 kernel: [15199.836659] 0000000000002260 000000004f5cf3c0 ffff880201a97000 0000000000000000 Aug 23 14:29:55 xen-001 kernel: [15199.837369] ffff880201a8b890 ffff88006f9ef000 ffff880008932200 ffffc90060ebfbf8 Aug 23 14:29:55 xen-001 kernel: [15199.838081] Call Trace: Aug 23 14:29:55 xen-001 kernel: [15199.838355] [<ffffffff8152e804>] n_tty_receive_buf2+0x14/0x20 Aug 23 14:29:55 xen-001 kernel: [15199.838627] [<ffffffff81531533>] tty_ldisc_receive_buf+0x23/0x50 Aug 23 14:29:55 xen-001 kernel: [15199.838900] [<ffffffff81531958>] flush_to_ldisc+0xc8/0x100 Aug 23 14:29:55 xen-001 kernel: [15199.839177] [<ffffffff8102eb3c>] ? __switch_to+0x1dc/0x680 Aug 23 14:29:55 xen-001 kernel: [15199.839454] [<ffffffff810c0490>] process_one_work+0x170/0x500 Aug 23 14:29:55 xen-001 kernel: [15199.839730] [<ffffffff818d4658>] ? __schedule+0x238/0x530 Aug 23 14:29:55 xen-001 kernel: [15199.840008] [<ffffffff818d4a3a>] ? schedule+0x3a/0xa0 Aug 23 14:29:55 xen-001 kernel: [15199.840280] [<ffffffff810c1426>] worker_thread+0x166/0x580 Aug 23 14:29:55 xen-001 kernel: [15199.840554] [<ffffffff810e6209>] ? put_prev_entity+0x29/0x140 Aug 23 14:29:55 xen-001 kernel: [15199.840826] [<ffffffff818d4658>] ? __schedule+0x238/0x530 Aug 23 14:29:55 xen-001 kernel: [15199.841099] [<ffffffff810d3882>] ? default_wake_function+0x12/0x20 Aug 23 14:29:55 xen-001 kernel: [15199.841373] [<ffffffff810c12c0>] ? maybe_create_worker+0x120/0x120 Aug 23 14:29:55 xen-001 kernel: [15199.841646] [<ffffffff818d4a3a>] ? schedule+0x3a/0xa0 Aug 23 14:29:55 xen-001 kernel: [15199.841919] [<ffffffff818d8826>] ? _raw_spin_unlock_irqrestore+0x16/0x20 Aug 23 14:29:55 xen-001 kernel: [15199.842193] [<ffffffff810c12c0>] ? maybe_create_worker+0x120/0x120 Aug 23 14:29:55 xen-001 kernel: [15199.842468] [<ffffffff810c62c5>] kthread+0xe5/0x100 Aug 23 14:29:55 xen-001 kernel: [15199.842741] [<ffffffff810c61e0>] ? __kthread_init_worker+0x40/0x40 Aug 23 14:29:55 xen-001 kernel: [15199.843017] [<ffffffff818d8f55>] ret_from_fork+0x25/0x30 Aug 23 14:29:55 xen-001 kernel: [15199.843288] Code: 89 fe 4c 89 ef 89 45 98 e8 aa fb ff ff 8b 45 98 48 63 d0 48 85 db 48 8d 0c 13 48 0f 45 d9 01 45 bc 49 01 d7 41 29 c4 48 8b 45 b0 <48> 8b 30 48 89 75 c0 49 8b 0e 8d 96 00 10 00 00$ Aug 23 14:29:55 xen-001 kernel: [15199.847901] RIP [<ffffffff8152e6a4>] n_tty_receive_buf_common+0xa4/0x1f0 Aug 23 14:29:55 xen-001 kernel: [15199.848244] RSP <ffffc90060ebfb08> Aug 23 14:29:55 xen-001 kernel: [15199.848511] CR2: 0000000000002260 Aug 23 14:29:55 xen-001 kernel: [15199.848781] ---[ end trace f98e9cf48e3a6111 ]--- Aug 23 14:29:55 xen-001 kernel: [15199.849242] BUG: unable to handle kernel paging request at ffffffffffffffd8 Aug 23 14:29:55 xen-001 kernel: [15199.849638] IP: [<ffffffff810c5aa0>] kthread_data+0x10/0x20 Aug 23 14:29:55 xen-001 kernel: [15199.849977] PGD 1e0a067 Aug 23 14:29:55 xen-001 kernel: [15199.850044] PUD 1e0c067
_______________________________________________ CentOS-virt mailing list CentOS-virt@xxxxxxxxxx https://lists.centos.org/mailman/listinfo/centos-virt
