also, this is x86-32 machine with a single Celeron processor 1.5Ghz. On Sun, Feb 5, 2012 at 11:57 AM, Raz <raziebe@xxxxxxxxx> wrote: > Hey Steven > > This is the first time I try frace on this kernel. it turned out that > when enabling > ftrace kernel oops whenever one loads a module - see bellow. > I debugged it a little and the null pointer is in trace_module_add_events > in the loop , the call pointer is NULL. > > for_each_event(call, start, end) { > call is null. > } > > I will continue debugging it a bit more. > > > 4.137324] e100 0000:02:03.0: eth0: NIC Link is Up 100 Mbps Full Duplex > [ 5.180354] Disabling lock debugging due to kernel taint > [ 5.180792] BUG: unable to handle kernel NULL pointer dereference at (null) > [ 5.180799] IP: [<c1084f7c>] trace_module_notify+0x9c/0x160 > [ 5.180810] *pde = 00000000 > [ 5.180813] Oops: 0000 [#1] PREEMPT > [ 5.180818] Modules linked in: sercos_driver(O+) > [ 5.180822] > [ 5.180826] Pid: 1079, comm: modprobe Tainted: G O > 3.2.0-rc5-rt8 #8 N/A N/A /MODB > [ 5.180832] EIP: 0060:[<c1084f7c>] EFLAGS: 00010286 CPU: 0 > [ 5.180836] EIP is at trace_module_notify+0x9c/0x160 > [ 5.180839] EAX: de513f44 EBX: dfeecc60 ECX: de513e0c EDX: de513e74 > [ 5.180842] ESI: 00000000 EDI: de513edc EBP: de60df3c ESP: de60df14 > [ 5.180846] DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 0068 > [ 5.180849] Process modprobe (pid: 1079, ti=de60c000 task=de519030 > task.ti=de60c000) > [ 5.180852] Stack: > [ 5.180854] de513e74 de513f44 de513edc de513f44 de513e0c de513e74 > 7fbb1b30 c14578b0 > [ 5.180859] 00000000 c14563a0 de60df58 c1055ba5 dfeecc60 00000001 > c1455d50 00000001 > [ 5.180865] dfeecc60 de60df78 c1055ee3 fffffffd 00000000 00000000 > dfeecc60 0000223f > [ 5.180870] Call Trace: > [ 5.180878] [<c1055ba5>] notifier_call_chain+0x45/0x60 > [ 5.180883] [<c1055ee3>] __blocking_notifier_call_chain+0x43/0x70 > [ 5.180888] [<c1055f2f>] blocking_notifier_call_chain+0x1f/0x30 > [ 5.180893] [<c1069682>] sys_init_module+0x62/0x1c0 > [ 5.180900] [<c10c32c1>] ? sys_close+0x71/0xc0 > [ 5.180906] [<c13441e1>] syscall_call+0x7/0xb > [ 5.180909] Code: 00 8d 90 44 01 00 00 83 c0 74 89 55 e4 89 45 ec > 8d b4 26 00 00 00 00 8b 45 e4 8b 55 ec 89 7c 24 08 8b 4d e8 89 44 24 > 04 89 14 24 > > > On Thu, Feb 2, 2012 at 4:12 PM, Steven Rostedt <rostedt@xxxxxxxxxxx> wrote: >> On Thu, 2012-02-02 at 12:52 +0200, Raz wrote: >>> hey >>> >>> I am trying to understand why a user space MT process behaves in an >>> unexpected manner. >>> I have real time process, which executes most of its threads in RT priority, >>> and from time to time a task ( medium priority ) is executing without >>> trying to stop. >>> >>> There are times, that this task is hogging the entire process and no >>> other **higher*** rt priority >>> task is getting any cpu time. >>> >>> Linux is **not** getting hogged. when setting the serial console ( >>> irq/serial ) and its shell ( /bin/sh ) >>> to a higher priority linux is responsive. >>> >>> Observing the task list through ps command i can see that this task's >>> threads when >>> to un-interruptible sleep. >>> >>> Any idea why is it happening ? >> >> Some ideas, but I need a lot more information. >> >>> kernel is 3.2.0-rc5-rt8. >> >> Is this a x86 box? >> >> Can you run a trace on the task that is going berserk? >> >> Enable CONFIG_FUNCTION_TRACER in your kernel and do the following: >> >> # mount -t debugfs nodev /sys/kernel/debug >> # cd /sys/kernel/debug/tracing >> # echo <pid-of-berserk-task> > set_ftrace_pid >> # echo function > current_tracer >> # cat trace >> >> Thanks! >> >> -- Steve >> >> -- To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html