On Wed, 20 Jul 2022 19:46:02 +0300 Alexey Dobriyan <adobriyan@xxxxxxxxx> wrote: > On Wed, Jul 20, 2022 at 12:11:02PM -0400, Steven Rostedt wrote: > > > > [ Adding live kernel patching folks ] > > > > On Wed, 20 Jul 2022 18:22:25 +0300 > > Alexey Dobriyan <adobriyan@xxxxxxxxx> wrote: > > > > > I'm debugging crash of our product which does live kernel patching together > > > with ISV security scanner which uses ftrace kprobes to do whatever it does. > > > > > > What happens is that is ftrace ever detects .text change, refuses to patch > > > and prints a warning with FTRACE_WARN_ON_ONCE() then there is reliable way > > > to cause UAF on kmalloc-512 cache by trying to register kprobe with > > > perf_event_open() and then unregistering it by exiting the process. > > > > > > 1) live kernel patching happens, first instruction of some function changes > > > > But live kernel patching uses ftrace to do this, so the question remains, > > why doesn't ftrace know about this change? > > That a line from our product which changes first instruction of the function. > So this is an out-of-tree change? > > > 2) kprobe on that function is registered with perf_event_open() > > > > > > WARNING: CPU: 5 PID: 2109 at kernel/trace/ftrace.c:1853 ftrace_bug+0x25d/0x270 > > > [<ffffffff811638ed>] ftrace_bug+0x25d/0x270 > > > [<ffffffff81065571>] ftrace_replace_code+0x2b1/0x420 > > > [<ffffffff81163f9a>] ftrace_modify_all_code+0x6a/0xb0 > > > [<ffffffff810656f0>] arch_ftrace_update_code+0x10/0x20 > > > [<ffffffff81164077>] ftrace_run_update_code+0x17/0x70 > > > [<ffffffff81165512>] ftrace_set_hash+0x1c2/0x1f0 > > > [<ffffffff8126fee0>] ? SyS_dup2+0x60/0x60 > > > [<ffffffff8126fee0>] ? SyS_dup2+0x60/0x60 > > > [<ffffffff811655a0>] ftrace_set_filter_ip+0x60/0x70 > > > [<ffffffff8179624c>] arm_kprobe+0x9c/0x140 > > > [<ffffffff81796368>] enable_kprobe+0x78/0xa0 > > > [<ffffffff81187bab>] enable_trace_kprobe+0x7b/0x120 > > > [<ffffffff81797e5f>] kprobe_register+0x2f/0x60 > > > [<ffffffff8118348a>] perf_trace_event_init+0x1aa/0x230 > > > [<ffffffff811836b7>] perf_kprobe_init+0xa7/0xf0 > > > [<ffffffff811a8919>] perf_kprobe_event_init+0x49/0x70 > > > [<ffffffff811aa569>] perf_try_init_event+0x99/0xc0 > > > [<ffffffff811b29f2>] perf_init_event+0x92/0x150 > > > [<ffffffff811b2fa1>] perf_event_alloc+0x4f1/0x910 > > > [<ffffffff811b3789>] SYSC_perf_event_open+0x3c9/0xe50 > > > [<ffffffff811b4679>] SyS_perf_event_open+0x9/0x10 > > > [<ffffffff81799f92>] system_call_fastpath+0x25/0x2a > > > ftrace failed to modify [<ffffffff8126fee0>] SyS_dup+0x0/0x120 > > > actual: e9:4b:50:2e:3f > > > > > > > Again, why did the above happen. This is a kernel bug that needs to be > > fixed. Everything else after this is unimportant, because it's just fall > > out to the above bug. > > I'll double check what we (not KLP) do and maybe even what KLP does! I would guess the fix is to have your stuff not monkey around with ftrace locations or to modify your own kernel to have ftrace know about this. -- Steve
