Am working on a problem where the system hanging after some unequal intervals, with the below o/p printing on the console,
Though this might be mostly tighten to the click router module am using, but I see the do_IRQ in the stack trace multiple times.
This brought some suspicious questions to me why so many times the module is interrupted.
Ealier I was running with centos5.3 kernel, now running with centos5.5 kernel (2.6.18-194).
In the new kernel config, I found CONFIG_PREPEMPT_NONE=y and CONFIG_PREEMPT_VOLUNTARY is not set, whereas these options are opposite in the old kernel.
This made to look for what preemption level the kernel is running with?
=======================
low stack detected by irq handler
[<c0405e9b>] do_IRQ+0x86/0xc3
[<c04046e6>] common_interrupt+0x1a/0x20
[<c05b83c6>] __ip_route_output_key+0x774/0x79e
[<fb817e32>] _ZN8FromHost5fl_txEP7sk_buffP10net_device+0xc2/0x100 [click]
[<c05a1b23>] dev_hard_start_xmit+0x1b4/0x25a
[<c05af12f>] __qdisc_run+0xd6/0x153
[<c05a3196>] dev_queue_xmit+0x1f8/0x216
[<c05a7431>] neigh_resolve_output+0x170/0x19a
[<c05b8403>] ip_route_output_flow+0x13/0x1d4
[<c05bd2db>] ip_push_pending_frames+0x31a/0x3a4
[<c05b85cd>] ip_route_output_key+0x9/0xb
[<c05bdd5d>] ip_send_reply+0xf8/0x209
[<c0418baf>] __wake_up+0x11/0x1a
[<c059b077>] sock_def_readable+0x26/0x4b
[<c05c9cf4>] tcp_data_queue+0x4c7/0x9a4
[<c05cfc56>] tcp_v4_send_reset+0x104/0x11c
[<c05d0879>] tcp_v4_do_rcv+0x240/0x269
[<c05d28cb>] tcp_v4_rcv+0x7f7/0x848
[<c05bb173>] ip_local_deliver+0x157/0x1d0
[<c05baff2>] ip_rcv+0x3cb/0x3f5
[<c05a141d>] __netif_receive_skb+0x3b3/0x3d5
[<fb81c21e>] _ZN6ToHost4pushEiP6Packet+0xee/0x210 [click]
[<fb83985d>] _ZN12EnsureEther24pushEiP6Packet+0x1d/0x20 [click]
[<fb82b895>] _ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
[<c05bb173>] ip_local_deliver+0x157/0x1d0
[<c043f334>] __do_IRQ+0x0/0x9f
[<c0405ecc>] do_IRQ+0xb7/0xc3
[<c04046e6>] common_interrupt+0x1a/0x20
[<fb80942d>] _ZN4IPRw7Mapping9make_pairEiRK8IPFlowIDS3_iiPS0_S4_+0xad/0xd0 [click]
[<fb872674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
[<fb86679e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
[<fb80acfb>] _ZN4IPRw7Pattern14create_mappingEiRK8IPFlowIDiiPNS_7MappingES5_RK7HashMapIS1_S5_E+0x1cb/0x1f0 [click]
[<fb872674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
[<fb86679e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
[<fb883ce3>] _ZN10IPRewriter13apply_patternEPN4IPRw7PatternEiRK8IPFlowIDii+0x153/0x160 [click]
[<fb883d9e>] _ZN10IPRewriter4pushEiP6Packet+0xae/0x260 [click]
[<fb855535>] _ZN11MacThrottle10l2PushProcEiP6Packet+0x55/0x3b0 [click]
[<fb82b895>] _ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
[<c04046e6>] common_interrupt+0x1a/0x20
[<c05a007b>] dev_seq_next+0xc/0x16
[<fb82b895>] _ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
[<c05bf06c>] ip_queue_xmit+0x3cb/0x401
[<c05a3196>] dev_queue_xmit+0x1f8/0x216
[<fb855b96>] _ZN11MacThrottle4pushEiP6Packet+0x306/0x1890 [click]
[<c05bf06c>] ip_queue_xmit+0x3cb/0x401
[<c0417f22>] enqueue_task+0x29/0x39
[<c043f334>] __do_IRQ+0x0/0x9f
[<c0405ecc>] do_IRQ+0xb7/0xc3
[<c04046e6>] common_interrupt+0x1a/0x20
[<fb7c0806>] _ZN7Element4pushEiP6Packet+0x26/0x40 [click]
[<fb872674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
[<fb7c0806>] _ZN7Element4pushEiP6Packet+0x26/0x40 [click]
[<fb872674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
[<fb859075>] _ZN17NtlmLoginDetector4pushEiP6Packet+0x65/0x15d0 [click]
[<fb88a881>] _ZN15RecycledSkbPool7recycleEP7sk_buff+0x31/0xf0 [click]
[<fb866fdb>] _ZN14PerfigoDiscard4pushEiP6Packet+0x4b/0x1c0 [click]
[<c0411f41>] smp_apic_timer_interrupt+0x32/0x45
[<c05bb173>] ip_local_deliver+0x157/0x1d0
[<fb872674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
[<fb843e86>] _ZN11HashIPTable4pushEiP6Packet+0xf6/0x2d0 [click]
[<fb86679e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
[<fb81c21e>] _ZN6ToHost4pushEiP6Packet+0xee/0x210 [click]
[<fb7fd904>] _ZN8IPFilter4pushEiP6Packet+0x64/0xf0 [click]
[<fb84f517>] _ZN12IPPortLookup4pushEiP6Packet+0x47/0x9e0 [click]
[<fb7bca6f>] _ZN6Packet19expensive_uniqueifyEiib+0x18f/0x1c0 [click]
[<fb8385fc>] _ZN10DnsHandler17is_packet_allowedEPK6Packeti+0x8c/0x5e0 [click]
[<fb8319d8>] _ZN8BWShaper4pushEiP6Packet+0x58/0xe90 [click]
[<fb8319d8>] _ZN8BWShaper4pushEiP6Packet+0x58/0xe90 [click]
[<fb8333a0>] _ZN10DnsHandler15is_host_matchedERK6VectorINS_9HostEntryEERK6StringRb+0x90/0x150 [click]
[<fb833753>] _ZN10DnsHandler13get_dns_queryEPKcS1_RS1_R17perfigo_dns_queryR6String+0x43/0x90 [click]
[<fb838bb8>] _ZN10DnsHandler4pushEiP6Packet+0x68/0xb00 [click]
[<fb835593>] _ZN10DnsHandler17process_dns_replyEPK6Packet+0x493/0x860 [click]
[<fb82b67b>] _ZN11ARPQuerier44pushEiP6Packet+0x2ab/0x2370 [click]
[<fb838bb8>] _ZN10DnsHandler4pushEiP6Packet+0x68/0xb00 [click]
[<fb872674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
[<fb84d34f>] _ZN13IPFilterGroup4pushEiP6Packet+0x19f/0xa80 [click]
[<fb81c21e>] _ZN6ToHost4pushEiP6Packet+0xee/0x210 [click]
[<fb7bcb42>] _ZN6Packet4makeEjPKhjj+0x32/0x100 [click]
[<fb83985d>] _ZN12EnsureEther24pushEiP6Packet+0x1d/0x20 [click]
[<fb82b895>] _ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
[<c043f334>] __do_IRQ+0x0/0x9f
[<c0405ecc>] do_IRQ+0xb7/0xc3
[<fb83e492>] _ZN13HashIPLookup34pushEiP6Packet+0x112/0x1c0 [click]
[<fb8516cf>] _ZN15LinearIPFilter24pushEiP6Packet+0x4f/0x3f0 [click]
[<fb872674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
[<fb80c1d1>] _ZN14LinearIPLookup4pushEiP6Packet+0x71/0x100 [click]
[<fb829a94>] _ZN11ARPQuerier49handle_ipEP6Packet+0x734/0xe30 [click]
[<fb848289>] _ZN14HashMACLookup54pushEiP6Packet+0x249/0x890 [click]
[<fb86679e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
[<c0417f22>] enqueue_task+0x29/0x39
[<c05c0000>] ip_setsockopt+0x184/0xa03
[<fb83f5b7>] _ZN13HashIPLookup54pushEiP6Packet+0xc7/0x110 [click]
[<fb8516cf>] _ZN15LinearIPFilter24pushEiP6Packet+0x4f/0x3f0 [click]
[<fb7fd904>] _ZN8IPFilter4pushEiP6Packet+0x64/0xf0 [click]
[<fb869516>] _ZN10Classifier4pushEiP6Packet+0x76/0xe0 [click]
[<fb84262e>] _ZN12HashIPTable24pushEiP6Packet+0x18e/0x580 [click]
[<fb84262e>] _ZN12HashIPTable24pushEiP6Packet+0x18e/0x580 [click]
[<fb82b67b>] _ZN11ARPQuerier44pushEiP6Packet+0x2ab/0x2370 [click]
[<c05d28cb>] tcp_v4_rcv+0x7f7/0x848
[<fb82ab12>] _ZN11ARPQuerier415handle_responseEP6Packet+0x32/0x8f0 [click]
[<fb84857a>] _ZN14HashMACLookup54pushEiP6Packet+0x53a/0x890 [click]
[<fb7fd904>] _ZN8IPFilter4pushEiP6Packet+0x64/0xf0 [click]
[<fb82b895>] _ZN11ARPQuerier44pushEiP6Packet+0x4c5/0x2370 [click]
[<c05a141d>] __netif_receive_skb+0x3b3/0x3d5
[<fb883d9e>] _ZN10IPRewriter4pushEiP6Packet+0xae/0x260 [click]
[<fb84262e>] _ZN12HashIPTable24pushEiP6Packet+0x18e/0x580 [click]
[<fb86679e>] _ZN6Passer4pushEiP6Packet+0x12e/0x860 [click]
[<fb84262e>] _ZN12HashIPTable24pushEiP6Packet+0x18e/0x580 [click]
[<fb872674>] _ZN11PaintSwitch4pushEiP6Packet+0x44/0xe0 [click]
[<c05a141d>] __netif_receive_skb+0x3b3/0x3d5
[<c043f334>] __do_IRQ+0x0/0x9f
[<c0405ecc>] do_IRQ+0xb7/0xc3
[<c043f334>] __do_IRQ+0x0/0x9f
[<c0405ecc>] do_IRQ+0xb7/0xc3
[<fb869516>] _ZN10Classifier4pushEiP6Packet+0x76/0xe0 [click]
[<fb7c0806>] _ZN7Element4pushEiP6Packet+0x26/0x40 [click]
[<fb8162a5>] _ZN10FromDevice8run_taskEP4Task+0xa5/0x120 [click]
[<fb7d7127>] _ZN12RouterThread6driverEv+0x177/0x290 [click]
[<fb887822>] _Z11click_schedPv+0x82/0x130 [click]
[<fb8877a0>] _Z11click_schedPv+0x0/0x130 [click]
[<c0404987>] kernel_thread_helper+0x7/0x10
--Sri
On Mon, Oct 24, 2011 at 10:08 AM, Greg KH <greg@xxxxxxxxx> wrote:
On Sun, Oct 23, 2011 at 11:20:03PM -0400, Greg Freemyer wrote:I do not think so.
> On Sun, Oct 23, 2011 at 1:34 PM, sri <bskmohan@xxxxxxxxx> wrote:
> > No, uname did not show anything.
> > Is there any way to get the kernel preemption mode, programatically?
> >
> > Thanks,
> > --Sri
> >
> > On Fri, Oct 21, 2011 at 6:41 PM, Daniel Baluta <daniel.baluta@xxxxxxxxx>
> > wrote:
> >>
> >> On Fri, Oct 21, 2011 at 2:28 PM, sri <bskmohan@xxxxxxxxx> wrote:
> >> > Hi,
> >> >
> >> > Am using kernel 2.6.18-195(centos 5.5).
> >> > My kernel configs have CONFIG_PREEMPT_NONE=7 and
> >> > "CONFIG_PREEMPT_VOLUNTERY
> >> > is not set".
> >> > How to check that preemption is really in place?
> >> > Is there any way to check my kernel is configured with what preemption
> >> > levels?
> >>
> >> Hmm, uname -a?
> I'm sure its in /sys somewhere.
As documented in Documentation/ABI/, so perhaps you can read there.
> Remember /sys is part of the official ABI.
Not all distros enable this :(
> Also, you see what your config look like for sure by looking at
> /proc/config.gz (that file is virtual, but shows the contents of how
> your config file was at compile time for the running kernel.
I think the question needs to really be stated, why, from userspace,
does it matter if preempt is enabled or not? This should never be
something that userspace cares about at all.
greg k-h
--
--
Krishna Mohan B
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@xxxxxxxxxxxxxxxxx http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies