Re: Kernel panic with bridged kvm guest

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 04/01/2011 06:57 PM, David Miller wrote:
From: Scot<lkml@xxxxxxxxxxxxx>
Date: Fri, 01 Apr 2011 16:14:41 -0500

This kernel panic occurs when running a kvm guest, usually with
minimal cpu/disk/network activity. What might I try to correct it?
Known kernel bug, fixed by commit:

--------------------
commit 6b1e960fdbd75dcd9bcc3ba5ff8898ff1ad30b6e
Author: Herbert Xu<herbert@xxxxxxxxxxxxxxxxxxx>
Date:   Fri Mar 18 05:27:28 2011 +0000

     bridge: Reset IPCB when entering IP stack on NF_FORWARD

     Whenever we enter the IP stack proper from bridge netfilter we
     need to ensure that the skb is in a form the IP stack expects
     it to be in.

     The entry point on NF_FORWARD did not meet the requirements of
     the IP stack, therefore leading to potential crashes/panics.

     This patch fixes the problem.

     Signed-off-by: Herbert Xu<herbert@xxxxxxxxxxxxxxxxxxx>
     Acked-by: Stephen Hemminger<shemminger@xxxxxxxxxx>
     Signed-off-by: David S. Miller<davem@xxxxxxxxxxxxx>

diff --git a/net/bridge/br_netfilter.c b/net/bridge/br_netfilter.c
index f97af559..008ff6c 100644
--- a/net/bridge/br_netfilter.c
+++ b/net/bridge/br_netfilter.c
@@ -739,6 +739,9 @@ static unsigned int br_nf_forward_ip(unsigned int hook, struct sk_buff *skb,
  		nf_bridge->mask |= BRNF_PKT_TYPE;
  	}

+	if (br_parse_ip_options(skb))
+		return NF_DROP;
+
  	/* The physdev module checks on this */
  	nf_bridge->mask |= BRNF_BRIDGED;
  	nf_bridge->physoutdev = skb->dev;

David, thanks for the reply. It looks like the kernel (built from git commit ecb78ab6f30106ab72a575a25b1cdfd1633b7ca2) has this patch applied. What could I try next?

The output below is from a second panic. In a third case, the panic occurred with no kvm guest. In all three cases, the Code section [4d 02 3c 03 ... 01 04 41 80 ] is identical.

------

[17320.625559] BUG: unable to handle kernel NULL pointer dereference at 00000000000000cc
[17320.633369] IP: [<ffffffff8129fb09>] ip_options_compile+0x1c1/0x435
[17320.639619] PGD 0
[17320.641626] Oops: 0000 [#1] SMP
[17320.644850] last sysfs file: /sys/devices/virtual/net/br0/bridge/topology_change_detected
[17320.652986] CPU 0
[17320.654812] Modules linked in: tun kvm_intel kvm bridge stp loop snd_pcm snd_timer snd soundcore evdev tpm_tis snd_page_alloc psmouse tpm i7core_edac processor ghes edac_core dcdbas thermal_sys tpm_bios serio_raw pcspkr power_meter button hed ext2 mbcache dm_mod raid1 md_mod sd_mod crc_t10dif usb_storage uas uhci_hcd mpt2sas scsi _transport_sas ehci_hcd raid_class igb scsi_mod usbcore dca bnx2 [last unloaded: scsi_wait_scan]
[17320.692775]
[17320.694255] Pid: 0, comm: swapper Not tainted 2.6.39-rc1+ #2 Dell Inc. PowerEdge R510/0DPRKF
[17320.795233] Stack:
[17320.797231] 0000000000000000 ffffffff81276893 0000000000000000 ffffffff817e6180 [17320.804625] 000000000000004e ffff88041d8a5600 ffff88041d8a5600 ffff8804020e8928 [17320.812019] ffff8804020e8900 ffff880404450000 ffff880403e7e84e ffff880404450000
[17320.819410] Call Trace:
[17320.821842] <IRQ>
[17320.823938]  [<ffffffff81276893>] ? netif_receive_skb+0x52/0x58
[17320.829831] [<ffffffffa0200e3a>] ? br_parse_ip_options+0x133/0x1a0 [bridge] [17320.836846] [<ffffffffa0201bd8>] ? br_nf_pre_routing+0x348/0x3cb [bridge]
[17320.843687]  [<ffffffff8119d89b>] ? cpumask_next_and+0x2b/0x3a
[17320.849491]  [<ffffffff81298493>] ? nf_iterate+0x41/0x7e
[17320.854778]  [<ffffffffa01fce89>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
[17320.861272]  [<ffffffffa01fce89>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
[17320.867765]  [<ffffffff81298543>] ? nf_hook_slow+0x73/0x114
[17320.873309]  [<ffffffffa01fce89>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
[17320.879804]  [<ffffffffa01fce89>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
[17320.886299]  [<ffffffffa01fce6f>] ? NF_HOOK.clone.4+0x3c/0x56 [bridge]
[17320.892793]  [<ffffffffa01fd1e5>] ? br_handle_frame+0x195/0x1ac [bridge]
[17320.899461] [<ffffffffa01fd050>] ? br_handle_frame_finish+0x1c7/0x1c7 [bridge]
[17320.906732]  [<ffffffff8127646f>] ? __netif_receive_skb+0x2a7/0x450
[17320.912966]  [<ffffffff81276893>] ? netif_receive_skb+0x52/0x58
[17320.918855]  [<ffffffff81276d95>] ? napi_gro_receive+0x1f/0x2f
[17320.924657]  [<ffffffff8127696a>] ? napi_skb_finish+0x1c/0x31
[17320.930378]  [<ffffffffa0226fcd>] ? igb_poll+0x6d9/0x9ee [igb]
[17320.936184]  [<ffffffff810937ef>] ? rcu_start_gp+0x80/0x220
[17320.941728]  [<ffffffff81276ec0>] ? net_rx_action+0xa4/0x1b1
[17320.947360]  [<ffffffff8104ad0a>] ? __do_softirq+0xb8/0x176
[17320.952905]  [<ffffffff81333a1c>] ? call_softirq+0x1c/0x30
[17320.958364]  [<ffffffff8100aa57>] ? do_softirq+0x3f/0x84
[17320.963647]  [<ffffffff8104af75>] ? irq_exit+0x3f/0x8f
[17320.968758]  [<ffffffff8100a793>] ? do_IRQ+0x85/0x9e
[17320.973698]  [<ffffffff8132c993>] ? common_interrupt+0x13/0x13
[17320.979499] <EOI>
[17320.981593]  [<ffffffff8106132c>] ? enqueue_hrtimer+0x3f/0x53
[17320.987312] [<ffffffffa0304417>] ? arch_local_irq_enable+0x7/0x8 [processor] [17320.994412] [<ffffffffa0304fdf>] ? acpi_idle_enter_bm+0x218/0x250 [processor]
[17321.001600]  [<ffffffff8125ded9>] ? menu_select+0x169/0x296
[17321.007143]  [<ffffffff8125cfe9>] ? cpuidle_idle_call+0xf4/0x17e
[17321.013117]  [<ffffffff81008298>] ? cpu_idle+0xa2/0xc4
[17321.018228]  [<ffffffff8169db60>] ? start_kernel+0x3b9/0x3c4
[17321.023858]  [<ffffffff8169d3c6>] ? x86_64_start_kernel+0x102/0x10f
[17321.030092] Code: 4d 02 3c 03 0f 86 59 02 00 00 0f b6 d0 44 39 ea 7f 32 83 c2 03 44 39 ea 0f 8f 45 02 00 00 48 85 db 74 18 48 8b 74 24 10 0f b6 c0 <8b> 96 cc 00 00 00 89 54 05 ff 41 80 4c 24 08 04 80 01 04 41 80
[17321.049431] RIP  [<ffffffff8129fb09>] ip_options_compile+0x1c1/0x435
[17321.055762]  RSP <ffff88042f203af0>
[17321.059230] CR2: 00000000000000cc
[17321.062848] ---[ end trace f1c95adde6496bcc ]---
[17321.067442] BUG: scheduling while atomic: swapper/0/0x10000100
[17321.073245] Modules linked in: tun kvm_intel kvm bridge stp loop snd_pcm snd_timer snd soundcore evdev tpm_tis snd_page_alloc psmouse tpm i7core_edac processor ghes edac_core dcdbas thermal_sys tpm_bios serio_raw pcspkr power_meter button hed ext2 mbcache dm_mod raid1 md_mod sd_mod crc_t10dif usb_storage uas uhci_hcd mpt2sas scsi_transport_sas ehci_hcd raid_class igb scsi_mod usbcore dca bnx2 [last unloaded: scsi_wait_scan]
[17321.111036] CPU 0
[17321.112862] Modules linked in: tun kvm_intel kvm bridge stp loop snd_pcm snd_timer snd soundcore evdev tpm_tis snd_page_alloc psmouse tpm i7core_edac processor ghes edac_core dcdbas thermal_sys tpm_bios serio_raw pcspkr power_meter button hed ext2 mbcache dm_mod raid1 md_mod sd_mod crc_t10dif usb_storage uas uhci_hcd mpt2sas scsi_transport_sas ehci_hcd raid_class igb scsi_mod usbcore dca bnx2 [last unloaded: scsi_wait_scan]
[17321.150828]
[17321.152309] Pid: 0, comm: swapper Tainted: G D 2.6.39-rc1+ #2 Dell Inc. PowerEdge R510/0DPRKF [17321.161597] RIP: 0010:[<ffffffffa0304417>] [<ffffffffa0304417>] arch_local_irq_enable+0x7/0x8 [processor]
[17321.171216] RSP: 0018:ffffffff81601eb0  EFLAGS: 00000292
[17321.176501] RAX: 00000000000310a6 RBX: ffffffff8106132c RCX: 00000000000003e8 [17321.183600] RDX: 0000000000000058 RSI: 0000000225c17d03 RDI: 000000000bf908c8 [17321.190700] RBP: ffff880403500000 R08: 00000000fffffffd R09: 0000000000000000 [17321.197799] R10: 00000000004d522d R11: 0000000000000246 R12: ffffffff8132c98e [17321.204901] R13: ffff88042f20feb0 R14: ffffffff811a2ede R15: ffff88042f20fdc8 [17321.212002] FS: 0000000000000000(0000) GS:ffff88042f200000(0000) knlGS:0000000000000000
[17321.220053] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[17321.225769] CR2: 00000000000000cc CR3: 0000000001603000 CR4: 00000000000006e0 [17321.232870] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [17321.239971] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [17321.247071] Process swapper (pid: 0, threadinfo ffffffff81600000, task ffffffff8160b020)
[17321.255122] Stack:
[17321.257121] ffffffffa0304fdf 0000000000011140 ffffffff8125ded9 0000000100411875 [17321.264514] ffff880403500020 ffff8804035000f0 0000000000000002 ffffffffffffffff [17321.271910] ffffffff8125cfe9 0000000000000000 ffffffff81600000 ffffffff816812d0
[17321.279307] Call Trace:
[17321.281739] [<ffffffffa0304fdf>] ? acpi_idle_enter_bm+0x218/0x250 [processor]
[17321.288927]  [<ffffffff8125ded9>] ? menu_select+0x169/0x296
[17321.294473]  [<ffffffff8125cfe9>] ? cpuidle_idle_call+0xf4/0x17e
[17321.300449]  [<ffffffff81008298>] ? cpu_idle+0xa2/0xc4
[17321.305562]  [<ffffffff8169db60>] ? start_kernel+0x3b9/0x3c4
[17321.311194]  [<ffffffff8169d3c6>] ? x86_64_start_kernel+0x102/0x10f
[17321.317429] Code: 63 1c fb 48 83 c4 38 89 e8 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 09 0f 1f 44 00 00 c3 fa 66 0f 1f 44 00 00 c3 fb 66 0f 1f 44 00 00 <c3> 48 8b 15 81 a0 41 e1 48 8d 42 fd 48 83 f8 01 0f 96 c0 48 ff
[17321.336774] Call Trace:
[17321.339207] [<ffffffffa0304fdf>] ? acpi_idle_enter_bm+0x218/0x250 [processor]
[17321.346394]  [<ffffffff8125ded9>] ? menu_select+0x169/0x296
[17321.351939]  [<ffffffff8125cfe9>] ? cpuidle_idle_call+0xf4/0x17e
[17321.357915]  [<ffffffff81008298>] ? cpu_idle+0xa2/0xc4
[17321.363028]  [<ffffffff8169db60>] ? start_kernel+0x3b9/0x3c4
[17321.368660]  [<ffffffff8169d3c6>] ? x86_64_start_kernel+0x102/0x10f
[17321.375364] Kernel panic - not syncing: Fatal exception in interrupt
[17321.381690] Pid: 0, comm: swapper Tainted: G      D     2.6.39-rc1+ #2
[17321.388185] Call Trace:
[17321.390615] <IRQ>  [<ffffffff8132ab34>] ? panic+0x92/0x1a1
[17321.396172]  [<ffffffff8132d6f6>] ? oops_end+0xa9/0xb6
[17321.401287]  [<ffffffff8102ca16>] ? no_context+0x1ed/0x1fa
[17321.406746]  [<ffffffff812aca41>] ? tcp_data_snd_check+0x3b/0x104
[17321.412810]  [<ffffffff8132f4e3>] ? do_page_fault+0x16b/0x308
[17321.418529]  [<ffffffff812b7ab7>] ? tcp_v4_do_rcv+0x172/0x31e
[17321.424247]  [<ffffffff812b7741>] ? sock_put+0xd/0x1c
[17321.429274]  [<ffffffff812b9d3a>] ? tcp_v4_rcv+0x4c7/0x66c
[17321.434732]  [<ffffffff8132cc55>] ? page_fault+0x25/0x30
[17321.440020]  [<ffffffffa01fce89>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
[17321.446517]  [<ffffffff8129fb09>] ? ip_options_compile+0x1c1/0x435
[17321.452667]  [<ffffffff81276893>] ? netif_receive_skb+0x52/0x58
[17321.458560] [<ffffffffa0200e3a>] ? br_parse_ip_options+0x133/0x1a0 [bridge] [17321.465577] [<ffffffffa0201bd8>] ? br_nf_pre_routing+0x348/0x3cb [bridge]
[17321.472418]  [<ffffffff8119d89b>] ? cpumask_next_and+0x2b/0x3a
[17321.478223]  [<ffffffff81298493>] ? nf_iterate+0x41/0x7e
[17321.483510]  [<ffffffffa01fce89>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
[17321.490007]  [<ffffffffa01fce89>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
[17321.496504]  [<ffffffff81298543>] ? nf_hook_slow+0x73/0x114
[17321.502051]  [<ffffffffa01fce89>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
[17321.508548]  [<ffffffffa01fce89>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
[17321.515044]  [<ffffffffa01fce6f>] ? NF_HOOK.clone.4+0x3c/0x56 [bridge]
[17321.521541]  [<ffffffffa01fd1e5>] ? br_handle_frame+0x195/0x1ac [bridge]
[17321.528212] [<ffffffffa01fd050>] ? br_handle_frame_finish+0x1c7/0x1c7 [bridge]
[17321.535487]  [<ffffffff8127646f>] ? __netif_receive_skb+0x2a7/0x450
[17321.541724]  [<ffffffff81276893>] ? netif_receive_skb+0x52/0x58
[17321.547613]  [<ffffffff81276d95>] ? napi_gro_receive+0x1f/0x2f
[17321.553418]  [<ffffffff8127696a>] ? napi_skb_finish+0x1c/0x31
[17321.559138]  [<ffffffffa0226fcd>] ? igb_poll+0x6d9/0x9ee [igb]
[17321.564943]  [<ffffffff810937ef>] ? rcu_start_gp+0x80/0x220
[17321.570489]  [<ffffffff81276ec0>] ? net_rx_action+0xa4/0x1b1
[17321.576122]  [<ffffffff8104ad0a>] ? __do_softirq+0xb8/0x176
[17321.581668]  [<ffffffff81333a1c>] ? call_softirq+0x1c/0x30
[17321.587126]  [<ffffffff8100aa57>] ? do_softirq+0x3f/0x84
[17321.592411]  [<ffffffff8104af75>] ? irq_exit+0x3f/0x8f
[17321.597524]  [<ffffffff8100a793>] ? do_IRQ+0x85/0x9e
[17321.602464]  [<ffffffff8132c993>] ? common_interrupt+0x13/0x13
[17321.608267] <EOI>  [<ffffffff8106132c>] ? enqueue_hrtimer+0x3f/0x53
[17321.614603] [<ffffffffa0304417>] ? arch_local_irq_enable+0x7/0x8 [processor] [17321.621704] [<ffffffffa0304fdf>] ? acpi_idle_enter_bm+0x218/0x250 [processor]
[17321.628892]  [<ffffffff8125ded9>] ? menu_select+0x169/0x296
[17321.634438]  [<ffffffff8125cfe9>] ? cpuidle_idle_call+0xf4/0x17e
[17321.640415]  [<ffffffff81008298>] ? cpu_idle+0xa2/0xc4
[17321.645529]  [<ffffffff8169db60>] ? start_kernel+0x3b9/0x3c4
[17321.651160]  [<ffffffff8169d3c6>] ? x86_64_start_kernel+0x102/0x10f
[17321.657397] BUG: scheduling while atomic: swapper/0/0x10000100
[17321.663200] Modules linked in: tun kvm_intel kvm bridge stp loop snd_pcm snd_timer snd soundcore evdev tpm_tis snd_page_alloc psmouse tpm i7core_edac processor ghes edac_core dcdbas thermal_sys tpm_bios serio_raw pcspkr power_meter button hed ext2 mbcache dm_mod raid1 md_mod sd_mod crc_t10dif usb_storage uas uhci_hcd mpt2sas scsi_transport_sas ehci_hcd raid_class igb scsi_mod usbcore dca bnx2 [last unloaded: scsi_wait_scan]
[17321.700982] CPU 0
[17321.702807] Modules linked in: tun kvm_intel kvm bridge stp loop snd_pcm snd_timer snd soundcore evdev tpm_tis snd_page_alloc psmouse tpm i7core_edac processor ghes edac_core dcdbas thermal_sys tpm_bios serio_raw pcspkr power_meter button hed ext2 mbcache dm_mod raid1 md_mod sd_mod crc_t10dif usb_storage uas uhci_hcd mpt2sas scsi_transport_sas ehci_hcd raid_class igb scsi_mod usbcore dca bnx2 [last unloaded: scsi_wait_scan]
[17321.740776]
[17321.742258] Pid: 0, comm: swapper Tainted: G D 2.6.39-rc1+ #2 Dell Inc. PowerEdge R510/0DPRKF [17321.751547] RIP: 0010:[<ffffffffa0304417>] [<ffffffffa0304417>] arch_local_irq_enable+0x7/0x8 [processor]
[17321.761166] RSP: 0018:ffffffff81601eb0  EFLAGS: 00000292
[17321.766450] RAX: 00000000000310a6 RBX: ffffffff8106132c RCX: 00000000000003e8 [17321.773551] RDX: 0000000000000058 RSI: 0000000225c17d03 RDI: 000000000bf908c8 [17321.780652] RBP: ffff880403500000 R08: 00000000fffffffd R09: 0000000000000000 [17321.787750] R10: 00000000004d522d R11: 0000000000000246 R12: ffffffff8132c98e [17321.794851] R13: ffff88042f20feb0 R14: ffffffff811a2ede R15: ffff88042f20fdc8 [17321.801952] FS: 0000000000000000(0000) GS:ffff88042f200000(0000) knlGS:0000000000000000
[17321.810003] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[17321.815720] CR2: 00007f422c0081a8 CR3: 0000000405992000 CR4: 00000000000006e0 [17321.822822] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [17321.829923] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [17321.837024] Process swapper (pid: 0, threadinfo ffffffff81600000, task ffffffff8160b020)
[17321.845076] Stack:
[17321.847075] ffffffffa0304fdf 0000000000011140 ffffffff8125ded9 0000000100411875 [17321.854470] ffff880403500020 ffff8804035000f0 0000000000000002 ffffffffffffffff [17321.861864] ffffffff8125cfe9 0000000000000000 ffffffff81600000 ffffffff816812d0
[17321.869259] Call Trace:
[17321.871692] [<ffffffffa0304fdf>] ? acpi_idle_enter_bm+0x218/0x250 [processor]
[17321.878879]  [<ffffffff8125ded9>] ? menu_select+0x169/0x296
[17321.884424]  [<ffffffff8125cfe9>] ? cpuidle_idle_call+0xf4/0x17e
[17321.890400]  [<ffffffff81008298>] ? cpu_idle+0xa2/0xc4
[17321.895513]  [<ffffffff8169db60>] ? start_kernel+0x3b9/0x3c4
[17321.901145]  [<ffffffff8169d3c6>] ? x86_64_start_kernel+0x102/0x10f
[17321.907380] Code: 63 1c fb 48 83 c4 38 89 e8 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 09 0f 1f 44 00 00 c3 fa 66 0f 1f 44 00 00 c3 fb 66 0f 1f 44 00 00 <c3> 48 8b 15 81 a0 41 e1 48 8d 42 fd 48 83 f8 01 0f 96 c0 48 ff
[17321.926724] Call Trace:
[17321.929156] [<ffffffffa0304fdf>] ? acpi_idle_enter_bm+0x218/0x250 [processor]
[17321.936343]  [<ffffffff8125ded9>] ? menu_select+0x169/0x296
[17321.941888]  [<ffffffff8125cfe9>] ? cpuidle_idle_call+0xf4/0x17e
[17321.947865]  [<ffffffff81008298>] ? cpu_idle+0xa2/0xc4
[17321.952978]  [<ffffffff8169db60>] ? start_kernel+0x3b9/0x3c4
[17321.958610]  [<ffffffff8169d3c6>] ? x86_64_start_kernel+0x102/0x10f
--
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Netdev]     [Ethernet Bridging]     [Linux 802.1Q VLAN]     [Linux Wireless]     [Kernel Newbies]     [Security]     [Linux for Hams]     [Netfilter]     [Git]     [Bugtraq]     [Yosemite News and Information]     [MIPS Linux]     [ARM Linux]     [Linux RAID]     [Linux PCI]     [Linux Admin]     [Samba]

  Powered by Linux