On Tue, Mar 16, 2021 at 02:00:07PM -0700, Alexei Starovoitov wrote:
> From: Alexei Starovoitov <ast@xxxxxxxxxx>
>
> The fexit/fmod_ret programs can be attached to kernel functions that can sleep.
> The synchronize_rcu_tasks() will not wait for such tasks to complete.
> In such case the trampoline image will be freed and when the task
> wakes up the return IP will point to freed memory causing the crash.
> Solve this by adding percpu_ref_get/put for the duration of trampoline
> and separate trampoline vs its image life times.
> The "half page" optimization has to be removed, since
> first_half->second_half->first_half transition cannot be guaranteed to
> complete in deterministic time. Every trampoline update becomes a new image.
> The image with fmod_ret or fexit progs will be freed via percpu_ref_kill and
> call_rcu_tasks. Together they will wait for the original function and
> trampoline asm to complete. The trampoline is patched from nop to jmp to skip
> fexit progs. They are freed independently from the trampoline. The image with
> fentry progs only will be freed via call_rcu_tasks_trace+call_rcu_tasks which
> will wait for both sleepable and non-sleepable progs to complete.
>
> Reported-by: Andrii Nakryiko <andrii@xxxxxxxxxx>
> Fixes: fec56f5890d9 ("bpf: Introduce BPF trampoline")
> Signed-off-by: Alexei Starovoitov <ast@xxxxxxxxxx>
> Acked-by: Paul E. McKenney <paulmck@xxxxxxxxxx> # for RCU
> ---
> Without ftrace fix:
> https://patchwork.kernel.org/project/netdevbpf/patch/20210316195815.34714-1-alexei.starovoitov@xxxxxxxxx/
> this patch will trigger warn in ftrace.
>
> arch/x86/net/bpf_jit_comp.c | 26 ++++-
> include/linux/bpf.h | 24 +++-
> kernel/bpf/bpf_struct_ops.c | 2 +-
> kernel/bpf/core.c | 4 +-
> kernel/bpf/trampoline.c | 218 +++++++++++++++++++++++++++---------
> 5 files changed, 213 insertions(+), 61 deletions(-)
>
hi,
I'm on bpf/master and I'm triggering warnings below when running together:
# while :; do ./test_progs -t fentry_test ; done
# while :; do ./test_progs -t module_attach ; done
when I revert this patch (plus b90829704780) it seems ok
jirka
---
[ 548.594548] bpf_testmod: loading out-of-tree module taints kernel.
[ 548.600787] bpf_testmod: module verification failed: signature and/or required key missing - tainting kernel
[ 558.353423] ------------[ cut here ]------------
[ 558.358064] WARNING: CPU: 35 PID: 1572 at kernel/bpf/syscall.c:2516 bpf_tracing_link_release+0x3b/0x40
[ 558.367399] Modules linked in: intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm ipmi_ssif irqbypass rapl mei_me iTCO_wdt intel_cstate wmi_bmof iTCO_ve]
[ 558.409989] CPU: 35 PID: 1572 Comm: test-66 Tainted: G IOE 5.12.0-rc2+ #25
[ 558.418005] Hardware name: Dell Inc. PowerEdge R440/08CYF7, BIOS 1.7.0 12/14/2018
[ 558.425492] RIP: 0010:bpf_tracing_link_release+0x3b/0x40
[ 558.430829] Code: 48 8b 7f 18 e8 26 5c 02 00 85 c0 75 1d 48 8b 7b 48 e8 29 53 02 00 48 8b 7b 50 48 85 ff 74 05 e8 bb f4 ff ff 48 8b 5d f8 c9 c3 <0f> 0b eb df 90 0f 1f 44 00 00 55 48 89 e5 41 54 4f
[ 558.449588] RSP: 0018:ffffc90002107e40 EFLAGS: 00010286
[ 558.454828] RAX: 00000000ffffffed RBX: ffff888105982300 RCX: 0000000000000000
[ 558.461969] RDX: ffff8881132c2540 RSI: 4c376cb4fcbc233e RDI: ffff8881058595d0
[ 558.469110] RBP: ffffc90002107e48 R08: 0000000000000000 R09: 00000000ffff850f
[ 558.476250] R10: 000000000000000a R11: 0000000000000008 R12: ffff888105982300
[ 558.483391] R13: ffff8881040743f8 R14: ffff8881039465a0 R15: ffff888141359440
[ 558.490532] FS: 00007f4232341740(0000) GS:ffff8897e10c0000(0000) knlGS:0000000000000000
[ 558.498625] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 558.504371] CR2: 0000000001ef164d CR3: 000000014b23a002 CR4: 00000000007706e0
[ 558.511505] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 558.518645] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 558.525778] PKRU: 55555554
[ 558.528492] Call Trace:
[ 558.530956] bpf_link_free+0x55/0x80
[ 558.534539] bpf_link_release+0x29/0x70
[ 558.538389] __fput+0x9f/0x250
[ 558.541457] ____fput+0xe/0x10
[ 558.544525] task_work_run+0x64/0xa0
[ 558.548112] exit_to_user_mode_prepare+0x11c/0x120
[ 558.552914] syscall_exit_to_user_mode+0x21/0x40
[ 558.557543] ? __x64_sys_close+0x12/0x40
[ 558.561476] do_syscall_64+0x45/0x50
[ 558.565065] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 558.570125] RIP: 0033:0x7f4232524167
[ 558.573713] Code: 00 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 41 c3 48 83 ec 18 89 7c 24 0f
[ 558.592470] RSP: 002b:00007ffd76f88ec8 EFLAGS: 00000246 ORIG_RAX: 0000000000000003
[ 558.600047] RAX: 0000000000000000 RBX: 0000000000000005 RCX: 00007f4232524167
[ 558.607187] RDX: 0000000005824950 RSI: 000000000582ec50 RDI: 0000000000000009
[ 558.614326] RBP: 000000000582ec80 R08: 0000000000000000 R09: 00007ffd76f88df7
[ 558.621466] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[ 558.628600] R13: 00000000000001c8 R14: 00000000fffffffb R15: 000000000000000b
[ 558.635741] ---[ end trace 878f3b01fdcfe925 ]---
[ 563.521703] ------------[ cut here ]------------
[ 563.526335] WARNING: CPU: 37 PID: 1586 at kernel/trace/ftrace.c:6321 ftrace_module_enable+0x33d/0x370
[ 563.535559] Modules linked in: bpf_testmod(OE+) intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm ipmi_ssif irqbypass rapl mei_me iTCO_wdt intel_cstate]
[ 563.579581] CPU: 37 PID: 1586 Comm: test_progs Tainted: G W IOE 5.12.0-rc2+ #25
[ 563.587862] Hardware name: Dell Inc. PowerEdge R440/08CYF7, BIOS 1.7.0 12/14/2018
[ 563.595348] RIP: 0010:ftrace_module_enable+0x33d/0x370
[ 563.600495] Code: 74 99 48 81 ca 00 00 00 10 49 89 54 24 08 e9 dc fe ff ff 8b 8b 98 01 00 00 48 01 ca 48 39 d0 0f 83 2e fd ff ff e9 65 fd ff ff <0f> 0b e9 be fe ff ff 0f 0b e9 b7 fe ff ff 48 83 7d
[ 563.619249] RSP: 0018:ffffc90002137d18 EFLAGS: 00010206
[ 563.624483] RAX: 0000000000031045 RBX: ffffffffa06793c0 RCX: 000000000000003d
[ 563.631617] RDX: ffff888108eb2080 RSI: ffffffffa06763c0 RDI: 0000000000000000
[ 563.638757] RBP: ffffc90002137d40 R08: ffffffff82b32ea0 R09: 0000000000000000
[ 563.645890] R10: 0000000000000001 R11: 0000000000000000 R12: ffff88810abd2030
[ 563.653027] R13: 61c8864680b583eb R14: 0000000000000003 R15: ffff888115d7d080
[ 563.660166] FS: 00007fcf8c7cb740(0000) GS:ffff8897e1140000(0000) knlGS:0000000000000000
[ 563.668259] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 563.674005] CR2: 000000000159d3d8 CR3: 000000010aad6001 CR4: 00000000007706e0
[ 563.681136] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 563.688270] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 563.695403] PKRU: 55555554
[ 563.698116] Call Trace:
[ 563.700573] load_module+0x2142/0x2610
[ 563.704333] __do_sys_finit_module+0xc2/0x120
[ 563.708700] __x64_sys_finit_module+0x1a/0x20
[ 563.713064] do_syscall_64+0x38/0x50
[ 563.716656] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 563.721715] RIP: 0033:0x7fcf8c8cc55d
[ 563.725302] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d eb 78 0c 00 f8
[ 563.744050] RSP: 002b:00007fffc6d14f38 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
[ 563.751625] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fcf8c8cc55d
[ 563.758764] RDX: 0000000000000000 RSI: 000000000159d3da RDI: 0000000000000004
[ 563.765899] RBP: 0000000000000004 R08: 0000000000000000 R09: 00007fffc6d83000
[ 563.773039] R10: 0000000000000003 R11: 0000000000000246 R12: 0000000000000000
[ 563.780174] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 563.787316] ---[ end trace 878f3b01fdcfe926 ]---
[ 563.903431] ------------[ cut here ]------------
[ 563.908057] WARNING: CPU: 32 PID: 1584 at kernel/trace/ftrace.c:1748 __ftrace_hash_rec_update.part.0+0x326/0x430
[ 563.918243] Modules linked in: bpf_testmod(OE) intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm ipmi_ssif irqbypass rapl mei_me iTCO_wdt intel_cstate ]
[ 563.962163] CPU: 32 PID: 1584 Comm: test-38 Tainted: G W IOE 5.12.0-rc2+ #25
[ 563.970172] Hardware name: Dell Inc. PowerEdge R440/08CYF7, BIOS 1.7.0 12/14/2018
[ 563.977658] RIP: 0010:__ftrace_hash_rec_update.part.0+0x326/0x430
[ 563.983757] Code: e7 c4 82 75 ca 49 8b 41 08 e9 30 ff ff ff 49 8b 41 08 4d 85 d2 0f 84 23 ff ff ff 48 0d 00 00 00 10 49 89 41 08 e9 63 fe ff ff <0f> 0b c7 05 2e fa aa 01 01 00 00 00 c7 05 34 fa a0
[ 564.002503] RSP: 0018:ffffc9000212fc48 EFLAGS: 00010246
[ 564.007731] RAX: 0000000000000000 RBX: ffff888115d7d080 RCX: 0000000000000001
[ 564.014873] RDX: 0000000000000003 RSI: ffffffffa06763c0 RDI: 0000000000000000
[ 564.022015] RBP: ffffc9000212fcc0 R08: 0000000000000001 R09: ffff88810abd2030
[ 564.029154] R10: ffff888108eb2080 R11: 0000000000000000 R12: 0000000000000000
[ 564.036288] R13: 0000000000000007 R14: 0000000000000000 R15: 0000000000000003
[ 564.043418] FS: 00007ff79bdeb740(0000) GS:ffff8897e1000000(0000) knlGS:0000000000000000
[ 564.051503] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 564.057250] CR2: 00007ff79c039000 CR3: 000000010baee002 CR4: 00000000007706e0
[ 564.064383] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 564.071515] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 564.078649] PKRU: 55555554
[ 564.081361] Call Trace:
[ 564.083821] ? udp_destruct_sock+0x140/0x140
[ 564.088107] ftrace_hash_rec_update_modify+0x1f/0x80
[ 564.093081] ftrace_hash_move_and_update_ops+0xcf/0x240
[ 564.098314] ? bpf_fentry_test1+0x10/0x10
[ 564.102336] ftrace_set_hash+0x121/0x1d0
[ 564.106275] ? 0xffffffffa052e000
[ 564.109599] ? bpf_fentry_test1+0x10/0x10
[ 564.113620] unregister_ftrace_direct+0x7a/0x200
[ 564.118247] ? bpf_fentry_test1+0x10/0x10
[ 564.122271] bpf_trampoline_update+0x31e/0x3f0
[ 564.126726] ? __radix_tree_delete+0x87/0xf0
[ 564.131005] bpf_trampoline_unlink_prog+0x9c/0x140
[ 564.135808] bpf_tracing_link_release+0x1a/0x40
[ 564.140347] bpf_link_free+0x55/0x80
[ 564.143935] bpf_link_release+0x29/0x70
[ 564.147784] __fput+0x9f/0x250
[ 564.150851] ____fput+0xe/0x10
[ 564.153911] task_work_run+0x64/0xa0
[ 564.157498] exit_to_user_mode_prepare+0x11c/0x120
[ 564.162299] syscall_exit_to_user_mode+0x21/0x40
[ 564.166928] ? __x64_sys_close+0x12/0x40
[ 564.170863] do_syscall_64+0x45/0x50
[ 564.174451] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 564.179510] RIP: 0033:0x7ff79bfce167
[ 564.183090] Code: 00 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 41 c3 48 83 ec 18 89 7c 24 0f
[ 564.201836] RSP: 002b:00007ffd8a9d85a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000003
[ 564.209403] RAX: 0000000000000000 RBX: 0000000000000001 RCX: 00007ff79bfce167
[ 564.216536] RDX: 0000000007274970 RSI: 0000000007276370 RDI: 000000000000000f
[ 564.223669] RBP: 00000000072763a0 R08: 0000000000000000 R09: 00007ffd8a9d84d7
[ 564.230801] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[ 564.237936] R13: 00007ff79c039030 R14: 0000000000000002 R15: 0000000000000000
[ 564.245077] ---[ end trace 878f3b01fdcfe927 ]---
[ 564.467617] ------------[ cut here ]------------
[ 564.472245] WARNING: CPU: 32 PID: 1584 at kernel/trace/ftrace.c:5228 unregister_ftrace_direct+0x1df/0x200
[ 564.481814] Modules linked in: bpf_testmod(OE) intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm ipmi_ssif irqbypass rapl mei_me iTCO_wdt intel_cstate ]
[ 564.525736] CPU: 32 PID: 1584 Comm: test-38 Tainted: G W IOE 5.12.0-rc2+ #25
[ 564.533745] Hardware name: Dell Inc. PowerEdge R440/08CYF7, BIOS 1.7.0 12/14/2018
[ 564.541233] RIP: 0010:unregister_ftrace_direct+0x1df/0x200
[ 564.546726] Code: 85 c0 75 0e 31 f6 48 c7 c7 a0 2e b3 82 e8 79 e4 ff ff 48 c7 c7 60 30 b3 82 e8 5d b1 94 00 e9 73 fe ff ff 0f 0b e9 1a ff ff ff <0f> 0b e9 a1 fe ff ff 0f 0b e9 0c ff ff ff 41 be e1
[ 564.565480] RSP: 0018:ffffc9000212fd90 EFLAGS: 00010286
[ 564.570716] RAX: 0000000000000001 RBX: ffffffffa0503000 RCX: ffffffff81954730
[ 564.577860] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff82b32ea0
[ 564.584996] RBP: ffffc9000212fdb8 R08: 0000000000000001 R09: 0000000000000000
[ 564.592128] R10: 0000000000000002 R11: ffff8881067cc790 R12: ffffffff81954730
[ 564.599261] R13: ffff888103ac18e0 R14: 00000000ffffffed R15: ffff888106251140
[ 564.606396] FS: 00007ff79bdeb740(0000) GS:ffff8897e1000000(0000) knlGS:0000000000000000
[ 564.614488] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 564.620235] CR2: 00007ff79c039000 CR3: 000000010baee002 CR4: 00000000007706e0
[ 564.627368] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 564.634500] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 564.641633] PKRU: 55555554
[ 564.644345] Call Trace:
[ 564.646800] ? bpf_fentry_test2+0x10/0x10
[ 564.650821] bpf_trampoline_update+0x31e/0x3f0
[ 564.655275] ? __radix_tree_delete+0x87/0xf0
[ 564.659558] bpf_trampoline_unlink_prog+0x9c/0x140
[ 564.664359] bpf_tracing_link_release+0x1a/0x40
[ 564.668900] bpf_link_free+0x55/0x80
[ 564.672486] bpf_link_release+0x29/0x70
[ 564.676333] __fput+0x9f/0x250
[ 564.679395] ____fput+0xe/0x10
[ 564.682462] task_work_run+0x64/0xa0
[ 564.686051] exit_to_user_mode_prepare+0x11c/0x120
[ 564.690869] syscall_exit_to_user_mode+0x21/0x40
[ 564.695502] ? __x64_sys_close+0x12/0x40
[ 564.699442] do_syscall_64+0x45/0x50
[ 564.703029] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 564.708089] RIP: 0033:0x7ff79bfce167
[ 564.711669] Code: 00 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 41 c3 48 83 ec 18 89 7c 24 0f
[ 564.730422] RSP: 002b:00007ffd8a9d85a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000003
[ 564.737989] RAX: 0000000000000000 RBX: 0000000000000002 RCX: 00007ff79bfce167
[ 564.745121] RDX: 0000000007274970 RSI: 00000000072763a0 RDI: 0000000000000010
[ 564.752254] RBP: 00000000072763d0 R08: 0000000000000000 R09: 00007ffd8a9d84d7
[ 564.759387] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
[ 564.766520] R13: 00007ff79c039030 R14: 0000000000000002 R15: 0000000000000000
[ 564.773655] ---[ end trace 878f3b01fdcfe928 ]---
[ 565.117608] ------------[ cut here ]------------
