On Fri, 28 Aug 2020 21:30:06 +0900 Masami Hiramatsu <mhiramat@xxxxxxxxxx> wrote: > From: Peter Zijlstra <peterz@xxxxxxxxxxxxx> > > The kretprobe hash is mostly superfluous, replace it with a per-task > variable. > > This gets rid of the task hash and it's related locking. > > The whole invalidate_rp_inst() is tedious and could go away once we > drop rp specific ri size. OK, something wrong with this patch. Now I can reproduce it always, it takes around 330 second from setting up the kretprobe event on schedule(). Maybe some timer or timeout value affect? Anyway, before this patch, I can not reproduce the bug yet. After applying this patch, I hit the bug always. So something wrong with this. ---- cd /sys/kernel/debug/tracing/ echo r:schedule schedule >> kprobe_events echo 1 > events/kprobes/enable sleep 333 ---- And show this. /sys/kernel/debug/tracing # [ 336.718043] ------------[ cut here ]------------ [ 336.719041] kernel BUG at kernel/kprobes.c:1950! [ 336.720212] invalid opcode: 0000 [#1] PREEMPT SMP PTI [ 336.721206] CPU: 0 PID: 85 Comm: kworker/0:2 Not tainted 5.9.0-rc2+ #58 [ 336.722557] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1 04/01/2014 [ 336.724310] Workqueue: 0x0 (events) [ 336.726997] RIP: 0010:__kretprobe_trampoline_handler+0xe5/0xf0 [ 336.728084] Code: e8 20 04 79 00 65 48 c7 05 ec 4a ec 7e 00 cd 28 82 4c 89 e7 e8 ac fe ff ff 48 85 db 75 97 5b 4c 89 e8 41 5c 41 5d 41 5e 5d c3 <0f> 0b 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 54 49 89 fc 53 48 [ 336.731574] RSP: 0018:ffffc90000307dc8 EFLAGS: 00010246 [ 336.732608] RAX: ffff88807d600000 RBX: 0000000000000000 RCX: 0000000000000000 [ 336.733896] RDX: ffffc90000307ea8 RSI: ffffffff810471e0 RDI: ffff88807c579700 [ 336.735205] RBP: ffffc90000307de8 R08: 0000000000000001 R09: 0000000000000001 [ 336.736506] R10: 0000000000000000 R11: 0000000000000001 R12: ffffc90000307e10 [ 336.737820] R13: ffff88807d62a440 R14: ffffc90000307e10 R15: ffff88807cfb5300 [ 336.739125] FS: 0000000000000000(0000) GS:ffff88807d600000(0000) knlGS:0000000000000000 [ 336.740705] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 336.741773] CR2: 000000000212cd38 CR3: 000000007a732000 CR4: 00000000000006b0 [ 336.742953] Call Trace: [ 336.743523] trampoline_handler+0x43/0x60 [ 336.744317] kretprobe_trampoline+0x2a/0x50 [ 336.745098] RIP: 0010:kretprobe_trampoline+0x0/0x50 [ 336.745961] Code: c7 39 2e 04 82 e8 a0 f2 0d 00 5d c3 31 f6 e9 79 ff ff ff be 01 00 00 00 e9 6f ff ff ff cc cc cc cc cc cc cc cc cc cc cc cc cc <54> 9c 48 83 ec 18 57 56 52 51 50 41 50 41 51 41 52 41 53 53 55 41 [ 336.749100] RSP: 7c579700:ffffc90000307eb0 EFLAGS: 00000246 [ 336.750077] RAX: ffff88807cfb5300 RBX: ffff88807d62a440 RCX: 0000000000000000 [ 336.751233] RDX: 0000000000000000 RSI: ffffffff818e5334 RDI: ffff88807c579700 [ 336.752391] RBP: ffffc90000307f00 R08: 0000000000000001 R09: 0000000000000001 [ 336.753563] R10: 0000000000000000 R11: 0000000000000001 R12: ffff88807cfb5328 [ 336.754718] R13: ffff88807d62a440 R14: ffffc90000033d58 R15: ffff88807cfb5300 [ 336.756005] ? schedule+0x54/0x100 [ 336.756684] ? process_one_work+0x5c0/0x5c0 [ 336.757504] kthread+0x13c/0x180 [ 336.758102] ? kthread_park+0x90/0x90 [ 336.758931] ret_from_fork+0x22/0x30 [ 336.759719] Modules linked in: [ 336.760456] ---[ end trace a7f6025840267136 ]--- [ 336.761448] RIP: 0010:__kretprobe_trampoline_handler+0xe5/0xf0 [ 336.762767] Code: e8 20 04 79 00 65 48 c7 05 ec 4a ec 7e 00 cd 28 82 4c 89 e7 e8 ac fe ff ff 48 85 db 75 97 5b 4c 89 e8 41 5c 41 5d 41 5e 5d c3 <0f> 0b 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 54 49 89 fc 53 48 [ 336.766128] RSP: 0018:ffffc90000307dc8 EFLAGS: 00010246 [ 336.767112] RAX: ffff88807d600000 RBX: 0000000000000000 RCX: 0000000000000000 [ 336.768321] RDX: ffffc90000307ea8 RSI: ffffffff810471e0 RDI: ffff88807c579700 [ 336.769539] RBP: ffffc90000307de8 R08: 0000000000000001 R09: 0000000000000001 [ 336.770744] R10: 0000000000000000 R11: 0000000000000001 R12: ffffc90000307e10 [ 336.771947] R13: ffff88807d62a440 R14: ffffc90000307e10 R15: ffff88807cfb5300 [ 336.773201] FS: 0000000000000000(0000) GS:ffff88807d600000(0000) knlGS:0000000000000000 [ 336.774706] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 336.775577] CR2: 000000000212cd38 CR3: 000000007a732000 CR4: 00000000000006b0 [ 336.776699] Kernel panic - not syncing: Fatal exception [ 336.777829] Kernel Offset: disabled [ 336.778526] ---[ end Kernel panic - not syncing: Fatal exception ]--- Thank you, -- Masami Hiramatsu <mhiramat@xxxxxxxxxx>
