I was using click-1.6.0 on centos 5.3 kernel and recently ported that to centos5.5 kernel..
After porting, am getting the (low stack detected by irq handler) and the following errors on the console frequently and causing the system unresponsive totally.
Even after reboot, this issue is re-occurring some time after every 3 days.
System info:
kernel: centos 5.5 (2.6.194-el5)
kernel Modules using: click router
NIC driver: tg3 (broadcom)
Note: The above setup was working successfully for years earlier.
I suspect tg3 driver is doing something here but not sure of how to confirm this.
Am looking for pointers to start with so that I can investigate. I tried to do web search on this but not useful.
Also, would like to know any ways/tools to debug using the addresses given in the stack trace.
Thanks in advance.
######################## START OF LOGS ############################
Oct 1 14:16:56 server03 kernel: low stack detected by irq handler
Oct 1 14:16:56 server03 kernel: [<c0405e9b>] do_IRQ+0x86/0xc3
Oct 1 14:16:56 server03 kernel: [<c04046e6>] common_interrupt+0x1a/0x20
Oct 1 14:16:56 server03 kernel: [<c05a5c06>] neigh_lookup+0x23/0x7f
Oct 1 14:16:56 server03 kernel: [<c05d6800>] arp_bind_neighbour+0x50/0x71
Oct 1 14:16:56 server03 kernel: [<c05b7b44>] rt_intern_hash+0x372/0x480
Oct 1 14:16:56 server03 kernel: [<c05b8352>] __ip_route_output_key+0x700/
0x79e
Oct 1 14:16:56 server03 kernel: [<c05b83d4>] __ip_route_output_key+0x782/0x79e
Oct 1 14:16:57 server03 kernel: [<c05b8403>] ip_route_output_flow+0x13/0x1d4
Oct 1 14:16:57 server03 kernel: [<c05b85cd>] ip_route_output_key+0x9/0xb
Oct 1 14:16:57 server03 kernel: [<c05bdd5d>] ip_send_reply+0xf8/0x209
Oct 1 14:16:57 server03 kernel: [<c045f1a6>] cache_alloc_refill+0x122/0x3e4
Oct 1 14:16:57 server03 kernel: [<c05b7c2f>] rt_intern_hash+0x45d/0x480
Oct 1 14:16:57 server03 kernel: [<c05cfc56>] tcp_v4_send_reset+0x104/0x11c
Oct 1 14:16:57 server03 kernel: [<c05d0879>] tcp_v4_do_rcv+0x240/0x269
Oct 1 14:16:57 server03 kernel: [<c05d28cb>] tcp_v4_rcv+0x7f7/0x848
Oct 1 14:16:57 server03 kernel: [<c05bb173>] ip_local_deliver+0x157/0x1d0
Oct 1 14:16:57 server03 kernel: [<c05baff2>] ip_rcv+0x3cb/0x3f5
Oct 1 14:16:57 server03 kernel: [<c05a141d>] __netif_receive_skb+0x3b3/0x3d5
Oct 1 14:16:57 server03 kernel: [<fb1b121e>] _ZN6ToHost4pushEiP6Packet+0xee/0x210 [click]
Oct 1 14:16:57 server03 kernel: [<fb1ce85d>] _ZN12EnsureEther24pushEiP6Packet+0x1d/0x20 [click]
Oct 1 14:16:57 server03 kernel: [<fb1c0895>] _ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
Oct 1 14:16:57 server03 kernel: [<c05bb173>] ip_local_deliver+0x157/0x1d0
Oct 1 14:16:57 server03 kernel: [<c05baff2>] ip_rcv+0x3cb/0x3f5
Oct 1 14:16:57 server03 kernel: [<fb19e42d>] _ZN4IPRw7Mapping9make_pairEiRK8IPFlowIDS3_iiPS0_S4_+0xad/0xd0 [click]
Oct 1 14:16:57 server03 kernel: [<fb19fcfb>] _ZN4IPRw7Pattern14create_mappingEiRK8IPFlowIDiiPNS_7MappingES5_RK7HashMapIS1_S5_E+0x1cb/0x1f0 [click]
Oct 1 14:16:57 server03 kernel: [<fb218ce3>] _ZN10IPRewriter13apply_patternEPN4IPRw7PatternEiRK8IPFlowIDii+0x153/0x160 [click]
Oct 1 14:16:57 server03 kernel: [<fb218d9e>] _ZN10IPRewriter4pushEiP6Packet+0xae/0x260 [click]
Oct 1 14:16:57 server03 kernel: [<fb1ea535>] _ZN11MacThrottle10l2PushProcEiP6Packet+0x55/0x3b0 [click]
Oct 1 14:16:57 server03 kernel: [<fb1c0895>] _ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
Oct 1 14:16:57 server03 kernel: [<fb218ce3>] _ZN10IPRewriter13apply_patternEPN4IPRw7PatternEiRK8IPFlowIDii+0x153/0x160 [click]
Oct 1 14:16:57 server03 kernel: [<fb218d9e>] _ZN10IPRewriter4pushEiP6Packet+0xae/0x260 [click]
Oct 1 14:16:58 server03 kernel: [<fb1eab96>] _ZN11MacThrottle4pushEiP6Packet+0x306/0x1890 [click]
Oct 1 14:16:58 server03 kernel: [<c0417f22>] enqueue_task+0x29/0x39
Oct 1 14:16:58 server03 kernel: [<c041805a>] __activate_task+0x4a/0x59
Oct 1 14:16:58 server03 kernel: [<fb1c69d8>] _ZN8BWShaper4pushEiP6Packet+0x58/0xe90 [click]
Oct 1 14:16:58 server03 kernel: [<fb155806>] _ZN7Element4pushEiP6Packet+0x26/0x40 [click]
Oct 1 14:16:58 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:58 server03 kernel: [<fb1ee075>] _ZN17NtlmLoginDetector4pushEiP6Packet+0x65/0x15d0 [click]
Oct 1 14:16:58 server03 kernel: [<fb155806>] _ZN7Element4pushEiP6Packet+0x26/0x40 [click]
Oct 1 14:16:58 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:58 server03 kernel: [<fb21f881>] _ZN15RecycledSkbPool7recycleEP7sk_buff+0x31/0xf0 [click]
Oct 1 14:16:58 server03 kernel: [<fb1fbfdb>] _ZN14PerfigoDiscard4pushEiP6Packet+0x4b/0x1c0 [click]
Oct 1 14:16:58 server03 kernel: [<c05bb173>] ip_local_deliver+0x157/0x1d0
Oct 1 14:16:58 server03 kernel: [<fb1d8e86>] _ZN11HashIPTable4pushEiP6Packet+0xf6/0x2d0 [click]
Oct 1 14:16:58 server03 kernel: [<c05a141d>] __netif_receive_skb+0x3b3/0x3d5
Oct 1 14:16:58 server03 kernel: [<fb1c69d8>] _ZN8BWShaper4pushEiP6Packet+0x58/0xe90 [click]
Oct 1 14:16:58 server03 kernel: [<fb192904>] _ZN8IPFilter4pushEiP6Packet+0x64/0xf0 [click]
Oct 1 14:16:58 server03 kernel: [<fb1e4517>] _ZN12IPPortLookup4pushEiP6Packet+0x47/0x9e0 [click]
Oct 1 14:16:58 server03 kernel: [<c043f334>] __do_IRQ+0x0/0x9f
Oct 1 14:16:58 server03 kernel: [<c0405ecc>] do_IRQ+0xb7/0xc3
Oct 1 14:16:58 server03 kernel: [<c043f334>] __do_IRQ+0x0/0x9f
Oct 1 14:16:58 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:58 server03 kernel: [<fb1cd5fc>] _ZN10DnsHandler17is_packet_allowedEPK6Packeti+0x8c/0x5e0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1c69d8>] _ZN8BWShaper4pushEiP6Packet+0x58/0xe90 [click]
Oct 1 14:16:59 server03 kernel: [<fb1c69d8>] _ZN8BWShaper4pushEiP6Packet+0x58/0xe90 [click]
Oct 1 14:16:59 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1cdbb8>] _ZN10DnsHandler4pushEiP6Packet+0x68/0xb00 [click]
Oct 1 14:16:59 server03 kernel: [<fb1b1853>] _ZN11ARPQuerier214send_query_forERK9IPAddressitit+0x133/0x3f0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1b1dba>] _ZN11ARPQuerier29handle_ipEP6Packet+0x2aa/0x5f0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1b31bd>] _ZN11ARPQuerier24pushEiP6Packet+0xcd/0x110 [click]
Oct 1 14:16:59 server03 kernel: [<fb1cdbb8>] _ZN10DnsHandler4pushEiP6Packet+0x68/0xb00 [click]
Oct 1 14:16:59 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1cdbb8>] _ZN10DnsHandler4pushEiP6Packet+0x68/0xb00 [click]
Oct 1 14:16:59 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1e234f>] _ZN13IPFilterGroup4pushEiP6Packet+0x19f/0xa80 [click]
Oct 1 14:16:59 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1fb79e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct 1 14:16:59 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:59 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1fb79e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct 1 14:16:59 server03 kernel: [<c05a5c59>] neigh_lookup+0x76/0x7f
Oct 1 14:16:59 server03 kernel: [<fb1d3492>] _ZN13HashIPLookup34pushEiP6Packet+0x112/0x1c0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1e66cf>] _ZN15LinearIPFilter24pushEiP6Packet+0x4f/0x3f0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1b80e8>] _ZN11ARPQuerier420handle_arp_retaggingEP6Packett+0x58/0x940 [click]
Oct 1 14:16:59 server03 kernel: [<c05bf82d>] ip_output+0x1f0/0x227
Oct 1 14:16:59 server03 kernel: [<fb1dd289>] _ZN14HashMACLookup54pushEiP6Packet+0x249/0x890 [click]
Oct 1 14:16:59 server03 kernel: [<fb1d45b7>] _ZN13HashIPLookup54pushEiP6Packet+0xc7/0x110 [click]
Oct 1 14:17:00 server03 kernel: [<fb1c0895>] _ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
Oct 1 14:17:00 server03 kernel: [<fb1fb79e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct 1 14:17:00 server03 kernel: [<fb1dcd23>] _ZN14HashMACLookup514push_extern_ipEP6Packet+0x1b3/0x4d0 [click]
Oct 1 14:17:00 server03 kernel: [<fb1d762e>] _ZN12HashIPTable24pushEiP6Packet+0x18e/0x580 [click]
Oct 1 14:17:00 server03 kernel: [<c05d065b>] tcp_v4_do_rcv+0x22/0x269
Oct 1 14:17:00 server03 kernel: [<c05d28cb>] tcp_v4_rcv+0x7f7/0x848
Oct 1 14:17:00 server03 kernel: [<fb1bfb12>] _ZN11ARPQuerier415handle_responseEP6Packet+0x32/0x8f0 [click]
Oct 1 14:17:00 server03 kernel: [<fb21f881>] _ZN15RecycledSkbPool7recycleEP7sk_buff+0x31/0xf0 [click]
Oct 1 14:17:00 server03 kernel: [<fb1fbfdb>] _ZN14PerfigoDiscard4pushEiP6Packet+0x4b/0x1c0 [click]
Oct 1 14:17:00 server03 kernel: [<fb192904>] _ZN8IPFilter4pushEiP6Packet+0x64/0xf0 [click]
Oct 1 14:17:00 server03 kernel: [<fb1c0895>] _ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
Oct 1 14:17:00 server03 kernel: [<fb1fb79e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct 1 14:17:00 server03 kernel: [<fb1fb79e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct 1 14:17:00 server03 kernel: [<fb1d762e>] _ZN12HashIPTable24pushEiP6Packet+0x18e/0x580 [click]
Oct 1 14:17:00 server03 kernel: [<c05d7777>] arp_rcv+0xd0/0xe1
Oct 1 14:17:00 server03 kernel: [<c05a141d>] __netif_receive_skb+0x3b3/0x3d5
Oct 1 14:17:00 server03 kernel: [<c043f334>] __do_IRQ+0x0/0x9f
Oct 1 14:17:00 server03 kernel: [<c043f334>] __do_IRQ+0x0/0x9f
Oct 1 14:17:00 server03 kernel: [<c0405ecc>] do_IRQ+0xb7/0xc3
Oct 1 14:17:00 server03 kernel: [<c0411f41>] smp_apic_timer_interrupt+0x32/0x45
Oct 1 14:17:00 server03 kernel: [<c040470b>] apic_timer_interrupt+0x1f/0x24
Oct 1 14:17:00 server03 kernel: [<c0405ecc>] do_IRQ+0xb7/0xc3
Oct 1 14:17:00 server03 kernel: [<c0405ecc>] do_IRQ+0xb7/0xc3
Oct 1 14:17:01 server03 kernel: [<fb1fe516>] _ZN10Classifier4pushEiP6Packet+0x76/0xe0 [click]
Oct 1 14:17:01 server03 kernel: [<fb155806>] _ZN7Element4pushEiP6Packet+0x26/0x40 [click]
Oct 1 14:17:01 server03 kernel: [<fb1ab2a5>] _ZN10FromDevice8run_taskEP4Task+0xa5/0x120 [click]
Oct 1 14:17:01 server03 kernel: [<fb16c127>] _ZN12RouterThread6driverEv+0x177/0x290 [click]
Oct 1 14:17:01 server03 kernel: [<fb21c822>] _Z11click_schedPv+0x82/0x130 [click]
Oct 1 14:17:01 server03 kernel: [<fb21c7a0>] _Z11click_schedPv+0x0/0x130 [click]
Oct 1 14:17:01 server03 kernel: [<c0404987>] kernel_thread_helper+0x7/0x10
Oct 1 14:17:01 server03 kernel: =======================
####################### END OF LOGS ######################################################
--
--
Krishna Mohan B
Oct 1 14:16:56 server03 kernel: [<c05b83d4>] __ip_route_output_key+0x782/0x79e
Oct 1 14:16:57 server03 kernel: [<c05b8403>] ip_route_output_flow+0x13/0x1d4
Oct 1 14:16:57 server03 kernel: [<c05b85cd>] ip_route_output_key+0x9/0xb
Oct 1 14:16:57 server03 kernel: [<c05bdd5d>] ip_send_reply+0xf8/0x209
Oct 1 14:16:57 server03 kernel: [<c045f1a6>] cache_alloc_refill+0x122/0x3e4
Oct 1 14:16:57 server03 kernel: [<c05b7c2f>] rt_intern_hash+0x45d/0x480
Oct 1 14:16:57 server03 kernel: [<c05cfc56>] tcp_v4_send_reset+0x104/0x11c
Oct 1 14:16:57 server03 kernel: [<c05d0879>] tcp_v4_do_rcv+0x240/0x269
Oct 1 14:16:57 server03 kernel: [<c05d28cb>] tcp_v4_rcv+0x7f7/0x848
Oct 1 14:16:57 server03 kernel: [<c05bb173>] ip_local_deliver+0x157/0x1d0
Oct 1 14:16:57 server03 kernel: [<c05baff2>] ip_rcv+0x3cb/0x3f5
Oct 1 14:16:57 server03 kernel: [<c05a141d>] __netif_receive_skb+0x3b3/0x3d5
Oct 1 14:16:57 server03 kernel: [<fb1b121e>] _ZN6ToHost4pushEiP6Packet+0xee/0x210 [click]
Oct 1 14:16:57 server03 kernel: [<fb1ce85d>] _ZN12EnsureEther24pushEiP6Packet+0x1d/0x20 [click]
Oct 1 14:16:57 server03 kernel: [<fb1c0895>] _ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
Oct 1 14:16:57 server03 kernel: [<c05bb173>] ip_local_deliver+0x157/0x1d0
Oct 1 14:16:57 server03 kernel: [<c05baff2>] ip_rcv+0x3cb/0x3f5
Oct 1 14:16:57 server03 kernel: [<fb19e42d>] _ZN4IPRw7Mapping9make_pairEiRK8IPFlowIDS3_iiPS0_S4_+0xad/0xd0 [click]
Oct 1 14:16:57 server03 kernel: [<fb19fcfb>] _ZN4IPRw7Pattern14create_mappingEiRK8IPFlowIDiiPNS_7MappingES5_RK7HashMapIS1_S5_E+0x1cb/0x1f0 [click]
Oct 1 14:16:57 server03 kernel: [<fb218ce3>] _ZN10IPRewriter13apply_patternEPN4IPRw7PatternEiRK8IPFlowIDii+0x153/0x160 [click]
Oct 1 14:16:57 server03 kernel: [<fb218d9e>] _ZN10IPRewriter4pushEiP6Packet+0xae/0x260 [click]
Oct 1 14:16:57 server03 kernel: [<fb1ea535>] _ZN11MacThrottle10l2PushProcEiP6Packet+0x55/0x3b0 [click]
Oct 1 14:16:57 server03 kernel: [<fb1c0895>] _ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
Oct 1 14:16:57 server03 kernel: [<fb218ce3>] _ZN10IPRewriter13apply_patternEPN4IPRw7PatternEiRK8IPFlowIDii+0x153/0x160 [click]
Oct 1 14:16:57 server03 kernel: [<fb218d9e>] _ZN10IPRewriter4pushEiP6Packet+0xae/0x260 [click]
Oct 1 14:16:58 server03 kernel: [<fb1eab96>] _ZN11MacThrottle4pushEiP6Packet+0x306/0x1890 [click]
Oct 1 14:16:58 server03 kernel: [<c0417f22>] enqueue_task+0x29/0x39
Oct 1 14:16:58 server03 kernel: [<c041805a>] __activate_task+0x4a/0x59
Oct 1 14:16:58 server03 kernel: [<fb1c69d8>] _ZN8BWShaper4pushEiP6Packet+0x58/0xe90 [click]
Oct 1 14:16:58 server03 kernel: [<fb155806>] _ZN7Element4pushEiP6Packet+0x26/0x40 [click]
Oct 1 14:16:58 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:58 server03 kernel: [<fb1ee075>] _ZN17NtlmLoginDetector4pushEiP6Packet+0x65/0x15d0 [click]
Oct 1 14:16:58 server03 kernel: [<fb155806>] _ZN7Element4pushEiP6Packet+0x26/0x40 [click]
Oct 1 14:16:58 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:58 server03 kernel: [<fb21f881>] _ZN15RecycledSkbPool7recycleEP7sk_buff+0x31/0xf0 [click]
Oct 1 14:16:58 server03 kernel: [<fb1fbfdb>] _ZN14PerfigoDiscard4pushEiP6Packet+0x4b/0x1c0 [click]
Oct 1 14:16:58 server03 kernel: [<c05bb173>] ip_local_deliver+0x157/0x1d0
Oct 1 14:16:58 server03 kernel: [<fb1d8e86>] _ZN11HashIPTable4pushEiP6Packet+0xf6/0x2d0 [click]
Oct 1 14:16:58 server03 kernel: [<c05a141d>] __netif_receive_skb+0x3b3/0x3d5
Oct 1 14:16:58 server03 kernel: [<fb1c69d8>] _ZN8BWShaper4pushEiP6Packet+0x58/0xe90 [click]
Oct 1 14:16:58 server03 kernel: [<fb192904>] _ZN8IPFilter4pushEiP6Packet+0x64/0xf0 [click]
Oct 1 14:16:58 server03 kernel: [<fb1e4517>] _ZN12IPPortLookup4pushEiP6Packet+0x47/0x9e0 [click]
Oct 1 14:16:58 server03 kernel: [<c043f334>] __do_IRQ+0x0/0x9f
Oct 1 14:16:58 server03 kernel: [<c0405ecc>] do_IRQ+0xb7/0xc3
Oct 1 14:16:58 server03 kernel: [<c043f334>] __do_IRQ+0x0/0x9f
Oct 1 14:16:58 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:58 server03 kernel: [<fb1cd5fc>] _ZN10DnsHandler17is_packet_allowedEPK6Packeti+0x8c/0x5e0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1c69d8>] _ZN8BWShaper4pushEiP6Packet+0x58/0xe90 [click]
Oct 1 14:16:59 server03 kernel: [<fb1c69d8>] _ZN8BWShaper4pushEiP6Packet+0x58/0xe90 [click]
Oct 1 14:16:59 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1cdbb8>] _ZN10DnsHandler4pushEiP6Packet+0x68/0xb00 [click]
Oct 1 14:16:59 server03 kernel: [<fb1b1853>] _ZN11ARPQuerier214send_query_forERK9IPAddressitit+0x133/0x3f0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1b1dba>] _ZN11ARPQuerier29handle_ipEP6Packet+0x2aa/0x5f0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1b31bd>] _ZN11ARPQuerier24pushEiP6Packet+0xcd/0x110 [click]
Oct 1 14:16:59 server03 kernel: [<fb1cdbb8>] _ZN10DnsHandler4pushEiP6Packet+0x68/0xb00 [click]
Oct 1 14:16:59 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1cdbb8>] _ZN10DnsHandler4pushEiP6Packet+0x68/0xb00 [click]
Oct 1 14:16:59 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1e234f>] _ZN13IPFilterGroup4pushEiP6Packet+0x19f/0xa80 [click]
Oct 1 14:16:59 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1fb79e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct 1 14:16:59 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:59 server03 kernel: [<fb207674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1fb79e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct 1 14:16:59 server03 kernel: [<c05a5c59>] neigh_lookup+0x76/0x7f
Oct 1 14:16:59 server03 kernel: [<fb1d3492>] _ZN13HashIPLookup34pushEiP6Packet+0x112/0x1c0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1e66cf>] _ZN15LinearIPFilter24pushEiP6Packet+0x4f/0x3f0 [click]
Oct 1 14:16:59 server03 kernel: [<fb1b80e8>] _ZN11ARPQuerier420handle_arp_retaggingEP6Packett+0x58/0x940 [click]
Oct 1 14:16:59 server03 kernel: [<c05bf82d>] ip_output+0x1f0/0x227
Oct 1 14:16:59 server03 kernel: [<fb1dd289>] _ZN14HashMACLookup54pushEiP6Packet+0x249/0x890 [click]
Oct 1 14:16:59 server03 kernel: [<fb1d45b7>] _ZN13HashIPLookup54pushEiP6Packet+0xc7/0x110 [click]
Oct 1 14:17:00 server03 kernel: [<fb1c0895>] _ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
Oct 1 14:17:00 server03 kernel: [<fb1fb79e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct 1 14:17:00 server03 kernel: [<fb1dcd23>] _ZN14HashMACLookup514push_extern_ipEP6Packet+0x1b3/0x4d0 [click]
Oct 1 14:17:00 server03 kernel: [<fb1d762e>] _ZN12HashIPTable24pushEiP6Packet+0x18e/0x580 [click]
Oct 1 14:17:00 server03 kernel: [<c05d065b>] tcp_v4_do_rcv+0x22/0x269
Oct 1 14:17:00 server03 kernel: [<c05d28cb>] tcp_v4_rcv+0x7f7/0x848
Oct 1 14:17:00 server03 kernel: [<fb1bfb12>] _ZN11ARPQuerier415handle_responseEP6Packet+0x32/0x8f0 [click]
Oct 1 14:17:00 server03 kernel: [<fb21f881>] _ZN15RecycledSkbPool7recycleEP7sk_buff+0x31/0xf0 [click]
Oct 1 14:17:00 server03 kernel: [<fb1fbfdb>] _ZN14PerfigoDiscard4pushEiP6Packet+0x4b/0x1c0 [click]
Oct 1 14:17:00 server03 kernel: [<fb192904>] _ZN8IPFilter4pushEiP6Packet+0x64/0xf0 [click]
Oct 1 14:17:00 server03 kernel: [<fb1c0895>] _ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
Oct 1 14:17:00 server03 kernel: [<fb1fb79e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct 1 14:17:00 server03 kernel: [<fb1fb79e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
Oct 1 14:17:00 server03 kernel: [<fb1d762e>] _ZN12HashIPTable24pushEiP6Packet+0x18e/0x580 [click]
Oct 1 14:17:00 server03 kernel: [<c05d7777>] arp_rcv+0xd0/0xe1
Oct 1 14:17:00 server03 kernel: [<c05a141d>] __netif_receive_skb+0x3b3/0x3d5
Oct 1 14:17:00 server03 kernel: [<c043f334>] __do_IRQ+0x0/0x9f
Oct 1 14:17:00 server03 kernel: [<c043f334>] __do_IRQ+0x0/0x9f
Oct 1 14:17:00 server03 kernel: [<c0405ecc>] do_IRQ+0xb7/0xc3
Oct 1 14:17:00 server03 kernel: [<c0411f41>] smp_apic_timer_interrupt+0x32/0x45
Oct 1 14:17:00 server03 kernel: [<c040470b>] apic_timer_interrupt+0x1f/0x24
Oct 1 14:17:00 server03 kernel: [<c0405ecc>] do_IRQ+0xb7/0xc3
Oct 1 14:17:00 server03 kernel: [<c0405ecc>] do_IRQ+0xb7/0xc3
Oct 1 14:17:01 server03 kernel: [<fb1fe516>] _ZN10Classifier4pushEiP6Packet+0x76/0xe0 [click]
Oct 1 14:17:01 server03 kernel: [<fb155806>] _ZN7Element4pushEiP6Packet+0x26/0x40 [click]
Oct 1 14:17:01 server03 kernel: [<fb1ab2a5>] _ZN10FromDevice8run_taskEP4Task+0xa5/0x120 [click]
Oct 1 14:17:01 server03 kernel: [<fb16c127>] _ZN12RouterThread6driverEv+0x177/0x290 [click]
Oct 1 14:17:01 server03 kernel: [<fb21c822>] _Z11click_schedPv+0x82/0x130 [click]
Oct 1 14:17:01 server03 kernel: [<fb21c7a0>] _Z11click_schedPv+0x0/0x130 [click]
Oct 1 14:17:01 server03 kernel: [<c0404987>] kernel_thread_helper+0x7/0x10
Oct 1 14:17:01 server03 kernel: =======================
####################### END OF LOGS ######################################################
--
--
Krishna Mohan B
--
--
Krishna Mohan B
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@xxxxxxxxxxxxxxxxx http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
