ftrace regression with CONFIG_UNWINDER_FRAME_POINTER

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I recently ran across an oops with ftrace when using the frame pointer
unwinder. It does not affect Linus' tree (tested 6.0-rc1), but it does
affect 5.15.60 and 5.10.136 (at least, have not tested other kernels).

It appears to be related to the recently merged retbleed mitigations,
in particular the unret mitigation. I originally reproduced it with
a live patch (livepatch-sample.ko can reproduce it) but can also
reproduce by setting a function filter with ftrace.

Oops from loading live patch:

[    0.672824] livepatch: 'livepatch_sample': patching complete
[    0.673809] BUG: unable to handle page fault for address: ffffffffc14dd888
[    0.674693] #PF: supervisor instruction fetch in kernel mode
[    0.675441] #PF: error_code(0x0010) - not-present page
[    0.676098] PGD 1ae10067 P4D 1ae10067 PUD 1ae12067 PMD 0
[    0.676790] Oops: 0010 [#1] SMP PTI
[    0.677239] CPU: 0 PID: 347 Comm: cat Tainted: G              K 5.15.60 #1
[    0.678127] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS ArchLinux 1.16.0-1 04/01/2014
[    0.679318] RIP: 0010:0xffffffffc14dd888
[    0.679820] Code: Unable to access opcode bytes at RIP 0xffffffffc14dd85e.
[    0.680689] RSP: 0018:ffffac644075bc68 EFLAGS: 00010282
[    0.681352] RAX: ffffffffa74d9cf0 RBX: 0000000000000000 RCX: 0000000000000001
[    0.682282] RDX: 0000000000400cc0 RSI: 0000000000000001 RDI: ffff970c8245f000
[    0.683184] RBP: ffffac644075bcc8 R08: 0000000000001000 R09: 0000000000000001
[    0.684081] R10: 0000000000000000 R11: ffff970c81868200 R12: ffffac644075bd20
[    0.684982] R13: ffffac644075bcf8 R14: 0000000000000001 R15: ffff970c8245f000
[    0.685925] FS:  0000000000713198(0000) GS:ffff970c9f400000(0000) knlGS:0000000000000000
[    0.686949] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    0.687677] CR2: ffffffffc14dd85e CR3: 000000000105c003 CR4: 0000000000370ef0
[    0.688579] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.689513] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[    0.690411] Call Trace:
[    0.690730]  <TASK>
[    0.691006]  ? 0xffffffffc03c9000
[    0.691425]  ? seq_read_iter+0x10e/0x440
[    0.691955]  proc_reg_read_iter+0x52/0x90
[    0.692487]  generic_file_splice_read+0xf1/0x190
[    0.693079]  do_splice_to+0x7d/0xc0
[    0.693528]  splice_direct_to_actor+0xbd/0x220
[    0.694100]  ? generic_file_splice_read+0x190/0x190
[    0.694723]  do_splice_direct+0x8a/0xd0
[    0.695214]  do_sendfile+0x352/0x4b0
[    0.695751]  __x64_sys_sendfile64+0xa9/0xc0
[    0.696285]  ? __context_tracking_exit+0x57/0x90
[    0.696868]  do_syscall_64+0x37/0x90
[    0.697324]  entry_SYSCALL_64_after_hwframe+0x61/0xcb
[    0.697964] RIP: 0033:0x4bf81e
[    0.698361] Code: de e1 fe 50 be 69 19 12 28 b8 a9 00 00 00 0f 05 48 89 c7 e8 84 0a f4 ff 5a c3 49 89 ca 48 63 f6 48 63 ff b8 28 00 00 00 0f 05 <48> 89 c7 e9 6a 0a f4 ff 50 b8 74 00 00 00 0f 05 48 89 c7 e8 5a 0a
[    0.700729] RSP: 002b:00007ffd027102b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000028
[    0.701670] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00000000004bf81e
[    0.702594] RDX: 0000000000000000 RSI: 0000000000000003 RDI: 0000000000000001
[    0.703483] RBP: 00007ffd02710310 R08: 0000000000000000 R09: 0000000001000000
[    0.704375] R10: 0000000001000000 R11: 0000000000000246 R12: 000000000000001b
[    0.705285] R13: 0000000000000001 R14: 0000000001000000 R15: 0000000000000001
[    0.706197]  </TASK>
[    0.706480] Modules linked in: livepatch_sample(K)
[    0.707085] CR2: ffffffffc14dd888
[    0.707508] ---[ end trace f6f689798d1c9538 ]---
[    0.708094] RIP: 0010:0xffffffffc14dd888
[    0.708610] Code: Unable to access opcode bytes at RIP 0xffffffffc14dd85e.
[    0.709480] RSP: 0018:ffffac644075bc68 EFLAGS: 00010282
[    0.710136] RAX: ffffffffa74d9cf0 RBX: 0000000000000000 RCX: 0000000000000001
[    0.711026] RDX: 0000000000400cc0 RSI: 0000000000000001 RDI: ffff970c8245f000
[    0.711915] RBP: ffffac644075bcc8 R08: 0000000000001000 R09: 0000000000000001
[    0.712864] R10: 0000000000000000 R11: ffff970c81868200 R12: ffffac644075bd20
[    0.713753] R13: ffffac644075bcf8 R14: 0000000000000001 R15: ffff970c8245f000
[    0.714649] FS:  0000000000713198(0000) GS:ffff970c9f400000(0000) knlGS:0000000000000000
[    0.715705] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    0.716426] CR2: ffffffffc14dd85e CR3: 000000000105c003 CR4: 0000000000370ef0
[    0.717317] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    0.718208] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Killed
[    0.720709] Kernel panic - not syncing: Attempted to kill init!  exitcode=0x00000000
[    0.721740] Kernel Offset: 0x26000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[    0.723133] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000000 ]---

Oops from setting ftrace filter:

[    0.644436] BUG: unable to handle page fault for address: ffffffffc1172888
[    0.645244] #PF: supervisor instruction fetch in kernel mode
[    0.645902] #PF: error_code(0x0010) - not-present page
[    0.646536] PGD 18e10067 P4D 18e10067 PUD 18e12067 PMD 0
[    0.647166] Oops: 0010 [#1] SMP PTI
[    0.647577] CPU: 0 PID: 1 Comm: init Not tainted 5.15.60 #1
[    0.648227] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS ArchLinux 1.16.0-1 04/01/2014
[    0.649284] BUG: unable to handle page fault for address: ffffffffc1172888
[    0.650117] #PF: supervisor instruction fetch in kernel mode
[    0.650772] #PF: error_code(0x0010) - not-present page
[    0.651370] PGD 18e10067 P4D 18e10067 PUD 18e12067 PMD 0
[    0.651998] Oops: 0010 [#2] SMP PTI
[    0.652409] CPU: 0 PID: 1 Comm: init Not tainted 5.15.60 #1
[    0.653059] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS ArchLinux 1.16.0-1 04/01/2014

Oddly this oops is incomplete but the portion that does get emitted
repeats a total of 4 times.

Requirements I've found to reproduce the oops:
- Recent LTS kernel (reproduced with 5.15.60 and 5.10.136 at least)
  - I could not cause the oops with Linus' tree (6.0-rc1 tested)
  - Have not tested other stable trees
- CONFIG_RETHUNK enabled
- CONFIG_UNWINDER_FRAME_POINTER enabled
  - CONFIG_UNWINDER_ORC does not cause an oops
- Using the default retbleed mitigations
  - Using retbleed=ibpb does not cause an oops on an AMD host

I've put together a quick reproducer using qemu. This tarball includes
a kernel and a minimal initrd that will reproduce the oops. Also
included is the .config used. I built this test kernel using gcc 12.1.0
(12.1.0-2) on an Arch Linux host but have also reproduced the oops using
gcc 9.3.0 (9.3.0-17ubuntu1~20.04) on an Ubuntu host.

http://johannes.erdfelt.com/ftrace-frame-pointer-regression.tar.gz

JE




[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Linux Kernel]     [Linux NFS]     [Linux NILFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux SCSI]

  Powered by Linux