[Bug report] NFS patch breaks TLS device-offloaded TX zerocopy

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

A recent patch [1] to 'fs' broke the TX TLS device-offloaded flow starting from v6.11-rc1.

The kernel crashes. Different runs result in different kernel traces.
See below [2].
All of them disappear once patch [1] is reverted.

The issues appears only with "sendfile on and zerocopy on".
We couldn't repro with "sendfile off", or with "sendfile on and zerocopy off".

The repro test is as simple as a repeated client/server communication (wrk/nginx), with sendfile on and zc on, and with "tls-hw-tx-offload: on".

$ for i in `seq 10`; do wrk -b::2:2:2:3 -t10 -c100 -d15 --timeout 5s https://[::2:2:2:2]:20448/16000b.img; done

We can provide more details if needed, to help with the analysis and debug.

Regards,
Tariq

[1]
commit 49b29a573da83b65d5f4ecf2db6619bab7aa910c
Author: Christoph Hellwig <hch@xxxxxx>
Date:   Mon May 27 18:36:09 2024 +0200

    nfs: add support for large folios

NFS already is void of folio size assumption, so just pass the chunk size to __filemap_get_folio and set the large folio address_space flag for all
    regular files.

    Signed-off-by: Christoph Hellwig <hch@xxxxxx>
    Tested-by: Sagi Grimberg <sagi@xxxxxxxxxxx>
    Signed-off-by: Anna Schumaker <Anna.Schumaker@xxxxxxxxxx>

 fs/nfs/file.c  | 4 +++-
 fs/nfs/inode.c | 1 +
 2 files changed, 4 insertions(+), 1 deletion(-)


[2]

Example #1:

rcu: INFO: rcu_sched self-detected stall on CPU
rcu: 0-....: (5249 ticks this GP) idle=cfb4/1/0x4000000000000000 softirq=1809/1813 fqs=2527
rcu:     (t=5250 jiffies g=2281 q=2004 ncpus=24)
CPU: 0 PID: 1047 Comm: nginx_openssl_3 Not tainted 6.10.0-bisect+ #21
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
RIP: 0010:xas_start+0x3f/0xc0
Code: 05 c0 ff ff 77 2d 48 8b 07 48 8b 57 08 48 8b 40 08 48 89 c1 83 e1 03 48 83 f9 02 75 08 48 3d 00 10 00 00 77 19 48 85 d2 75 21 <48> c7 47 18 00 00 00 00 c3 48 c1 fa 02 85 d2 74 cb 31 c0 c3 0f b6
RSP: 0018:ffff888108a4bad8 EFLAGS: 00000293
RAX: ffff88810c236912 RBX: ffff888108a4bc58 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff888108a4bae8
RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff888103d30318
R13: ffff8881002a3700 R14: 0000000000000000 R15: ffff888105ba2e40
FS:  00007fa598930740(0000) GS:ffff88885f800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055dd8d91fca0 CR3: 0000000108b7e005 CR4: 0000000000370eb0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 ? rcu_dump_cpu_stacks+0xc7/0x100
 ? rcu_sched_clock_irq+0x516/0xb20
 ? update_process_times+0x69/0xa0
 ? tick_nohz_handler+0x87/0x110
 ? tick_do_update_jiffies64+0xd0/0xd0
 ? __hrtimer_run_queues+0x121/0x270
 ? hrtimer_interrupt+0x10f/0x260
 ? __sysvec_apic_timer_interrupt+0x4f/0x110
 ? sysvec_apic_timer_interrupt+0x6c/0x90
 </IRQ>
 <TASK>
 ? asm_sysvec_apic_timer_interrupt+0x16/0x20
 ? xas_start+0x3f/0xc0
 xas_load+0x5/0xa0
 filemap_get_read_batch+0x19e/0x2a0
 filemap_get_pages+0x97/0x600
 ? nfs_update_inode+0x4b9/0xb70
 filemap_splice_read+0x12b/0x300
 ? tls_push_sg+0x13e/0x220
 ? tls_push_data+0x6bd/0xa40
 nfs_file_splice_read+0x78/0xa0
 splice_direct_to_actor+0xb0/0x230
 ? splice_file_range_actor+0x40/0x40
 do_splice_direct+0x73/0xb0
 ? propagate_umount+0x560/0x560
 do_sendfile+0x33b/0x3e0
 __x64_sys_sendfile64+0x5d/0xd0
 do_syscall_64+0x4c/0x100
 entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP: 0033:0x7fa598705dae
Code: c3 0f 1f 00 4c 89 d2 4c 89 c6 e9 fd fd ff ff 0f 1f 44 00 00 31 c0 c3 0f 1f 44 00 00 f3 0f 1e fa 49 89 ca b8 28 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4a 40 0f 00 f7 d8 64 89 01 48
RSP: 002b:00007ffc17804728 EFLAGS: 00000206 ORIG_RAX: 0000000000000028
RAX: ffffffffffffffda RBX: 0000000039960ce0 RCX: 00007fa598705dae
RDX: 00007ffc17804738 RSI: 0000000000000030 RDI: 0000000000000020
RBP: 0000000000000030 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000003e80 R11: 0000000000000206 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000003e80 R15: 00000000399b8a68
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 1048 Comm: nginx_openssl_3 Not tainted 6.10.0-bisect+ #21
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
RIP: 0010:xas_load+0x5/0xa0
Code: 48 c1 e8 02 0f b6 c0 48 83 c0 04 48 8b 44 c2 08 c3 48 8b 07 48 8b 40 08 c3 66 66 2e 0f 1f 84 00 00 00 00 00 90 e8 3b ff ff ff <48> 89 c2 83 e2 03 48 83 fa 02 75 08 48 3d 00 10 00 00 77 01 c3 0f
RSP: 0018:ffff888108a3bae0 EFLAGS: 00000293
RAX: ffff88810c236912 RBX: ffff888108a3bc58 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff888108a3bae8
RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff888103d30318
R13: ffff888103b14700 R14: 0000000000000000 R15: ffff888105fbbc80
FS:  00007fa598930740(0000) GS:ffff88885f840000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe83bffd550 CR3: 0000000108f09002 CR4: 0000000000370eb0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 ? nmi_cpu_backtrace+0x7f/0xe0
 ? nmi_cpu_backtrace_handler+0xd/0x20
 ? nmi_handle+0x56/0x150
 ? default_do_nmi+0x3e/0xd0
 ? exc_nmi+0xd8/0x100
 ? end_repeat_nmi+0xf/0x18
 ? xas_load+0x5/0xa0
 ? xas_load+0x5/0xa0
 ? xas_load+0x5/0xa0
 </NMI>
 <TASK>
 filemap_get_read_batch+0x19e/0x2a0
 filemap_get_pages+0x97/0x600
 ? nfs_update_inode+0x4b9/0xb70
 filemap_splice_read+0x12b/0x300
 ? tls_push_sg+0x13e/0x220
 ? tls_push_data+0x6bd/0xa40
 nfs_file_splice_read+0x78/0xa0
 splice_direct_to_actor+0xb0/0x230
 ? splice_file_range_actor+0x40/0x40
 do_splice_direct+0x73/0xb0
 ? propagate_umount+0x560/0x560
 do_sendfile+0x33b/0x3e0
 __x64_sys_sendfile64+0x5d/0xd0
 do_syscall_64+0x4c/0x100
 entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP: 0033:0x7fa598705dae
Code: c3 0f 1f 00 4c 89 d2 4c 89 c6 e9 fd fd ff ff 0f 1f 44 00 00 31 c0 c3 0f 1f 44 00 00 f3 0f 1e fa 49 89 ca b8 28 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4a 40 0f 00 f7 d8 64 89 01 48
RSP: 002b:00007ffc17804728 EFLAGS: 00000206 ORIG_RAX: 0000000000000028
RAX: ffffffffffffffda RBX: 000000003993d090 RCX: 00007fa598705dae
RDX: 00007ffc17804738 RSI: 000000000000002d RDI: 0000000000000019
RBP: 000000000000002d R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000003e80 R11: 0000000000000206 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000003e80 R15: 000000003999f4d8
 </TASK>
Sending NMI from CPU 0 to CPUs 2:
NMI backtrace for cpu 2
CPU: 2 PID: 1049 Comm: nginx_openssl_3 Not tainted 6.10.0-bisect+ #21
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
RIP: 0010:xas_load+0x53/0xa0
Code: 77 08 48 d3 ee 83 e6 3f 89 f0 48 83 c0 04 48 8b 44 c2 08 48 89 57 18 48 89 c1 83 e1 03 48 83 f9 02 74 10 40 88 77 12 80 3a 00 <75> b0 c3 48 83 f9 02 75 f0 48 3d fd 00 00 00 77 e8 48 c1 e8 02 89
RSP: 0018:ffff888103813ae0 EFLAGS: 00000246
RAX: ffffea00046ee800 RBX: ffff888103813c58 RCX: 0000000000000000
RDX: ffff88810c236910 RSI: 0000000000000000 RDI: ffff888103813ae8
RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff888103d30318
R13: ffff888103b08700 R14: 0000000000000000 R15: ffff888117432480
FS:  00007fa598930740(0000) GS:ffff88885f880000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7704001950 CR3: 000000010d823002 CR4: 0000000000370eb0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 ? nmi_cpu_backtrace+0x7f/0xe0
 ? nmi_cpu_backtrace_handler+0xd/0x20
 ? nmi_handle+0x56/0x150
 ? default_do_nmi+0x3e/0xd0
 ? exc_nmi+0xd8/0x100
 ? end_repeat_nmi+0xf/0x18
 ? xas_load+0x53/0xa0
 ? xas_load+0x53/0xa0
 ? xas_load+0x53/0xa0
 </NMI>
 <TASK>
 filemap_get_read_batch+0x19e/0x2a0
 filemap_get_pages+0x97/0x600
 ? nfs_update_inode+0x4b9/0xb70
 filemap_splice_read+0x12b/0x300
 ? tls_push_sg+0x13e/0x220
 ? tls_push_data+0x6bd/0xa40
 nfs_file_splice_read+0x78/0xa0
 splice_direct_to_actor+0xb0/0x230
 ? splice_file_range_actor+0x40/0x40
 do_splice_direct+0x73/0xb0
 ? propagate_umount+0x560/0x560
 do_sendfile+0x33b/0x3e0
 __x64_sys_sendfile64+0x5d/0xd0
 do_syscall_64+0x4c/0x100
 entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP: 0033:0x7fa598705dae
Code: c3 0f 1f 00 4c 89 d2 4c 89 c6 e9 fd fd ff ff 0f 1f 44 00 00 31 c0 c3 0f 1f 44 00 00 f3 0f 1e fa 49 89 ca b8 28 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4a 40 0f 00 f7 d8 64 89 01 48
RSP: 002b:00007ffc17804728 EFLAGS: 00000206 ORIG_RAX: 0000000000000028
RAX: ffffffffffffffda RBX: 0000000039906100 RCX: 00007fa598705dae
RDX: 00007ffc17804738 RSI: 0000000000000034 RDI: 000000000000001c
RBP: 0000000000000034 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000003e80 R11: 0000000000000206 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000003e80 R15: 00000000399b3888
 </TASK>
Sending NMI from CPU 0 to CPUs 3:
NMI backtrace for cpu 3
CPU: 3 PID: 1050 Comm: nginx_openssl_3 Not tainted 6.10.0-bisect+ #21
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
RIP: 0010:xas_start+0x53/0xc0
Code: 83 e1 03 48 83 f9 02 75 08 48 3d 00 10 00 00 77 19 48 85 d2 75 21 48 c7 47 18 00 00 00 00 c3 48 c1 fa 02 85 d2 74 cb 31 c0 c3 <0f> b6 48 fe 48 d3 ea 48 83 fa 3f 76 df 48 c7 47 18 01 00 00 00 31
RSP: 0018:ffff8881328dbad8 EFLAGS: 00000286
RAX: ffff88810c236912 RBX: ffff8881328dbc58 RCX: 0000000000000002
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8881328dbae8
RBP: 0000000000000003 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: ffff888103d30318
R13: ffff88810402e100 R14: 0000000000000000 R15: ffff888104032780
FS:  00007fa598930740(0000) GS:ffff88885f8c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055ddbea4a678 CR3: 0000000108b12001 CR4: 0000000000370eb0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 ? nmi_cpu_backtrace+0x7f/0xe0
 ? nmi_cpu_backtrace_handler+0xd/0x20
 ? nmi_handle+0x56/0x150
 ? default_do_nmi+0x3e/0xd0
 ? exc_nmi+0xd8/0x100
 ? end_repeat_nmi+0xf/0x18
 ? xas_start+0x53/0xc0
 ? xas_start+0x53/0xc0
 ? xas_start+0x53/0xc0
 </NMI>
 <TASK>
 xas_load+0x5/0xa0
 filemap_get_read_batch+0x19e/0x2a0
 filemap_get_pages+0x97/0x600
 ? nfs_update_inode+0x4b9/0xb70
 filemap_splice_read+0x12b/0x300
 ? tls_push_sg+0x13e/0x220
 ? common_interrupt+0xf/0xa0
 ? asm_common_interrupt+0x22/0x40
 ? _raw_spin_lock+0x10/0x20
 nfs_file_splice_read+0x78/0xa0
 splice_direct_to_actor+0xb0/0x230
 ? splice_file_range_actor+0x40/0x40
 do_splice_direct+0x73/0xb0
 ? propagate_umount+0x560/0x560
 do_sendfile+0x33b/0x3e0
 __x64_sys_sendfile64+0x5d/0xd0
 do_syscall_64+0x4c/0x100
 entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP: 0033:0x7fa598705dae
Code: c3 0f 1f 00 4c 89 d2 4c 89 c6 e9 fd fd ff ff 0f 1f 44 00 00 31 c0 c3 0f 1f 44 00 00 f3 0f 1e fa 49 89 ca b8 28 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4a 40 0f 00 f7 d8 64 89 01 48
RSP: 002b:00007ffc17804728 EFLAGS: 00000206 ORIG_RAX: 0000000000000028
RAX: ffffffffffffffda RBX: 000000003994a6d0 RCX: 00007fa598705dae
RDX: 00007ffc17804738 RSI: 000000000000002f RDI: 0000000000000016
RBP: 000000000000002f R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000003e80 R11: 0000000000000206 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000003e80 R15: 000000003998d548
 </TASK>


Example #2:

Oops: general protection fault, probably for non-canonical address 0xdead000000000122: 0000 [#1] SMP
CPU: 4 PID: 0 Comm: swapper/4 Not tainted 6.10.0-bisect+ #23
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
RIP: 0010:free_pcppages_bulk+0x12f/0x1e0
Code: 89 34 24 e8 a3 ed ff ff 49 8b 14 24 45 31 c9 4c 89 ff 49 89 c0 89 44 24 20 49 8b 44 24 08 8b 4c 24 0c 48 8b 34 24 48 89 42 08 <48> 89 10 48 8b 54 24 18 48 b8 00 01 00 00 00 00 ad de 49 89 04 24
RSP: 0018:ffff88885f905888 EFLAGS: 00010046
RAX: dead000000000122 RBX: ffff88885f932810 RCX: 0000000000000000
RDX: ffff88885f932830 RSI: 00000000001144a0 RDI: ffffea0004512800
RBP: 0000000000000001 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000001 R11: dead000000000100 R12: ffffea0004512808
R13: 000000000000003a R14: ffff88885f932800 R15: ffffea0004512800
FS:  0000000000000000(0000) GS:ffff88885f900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000056063c04b2e8 CR3: 000000000282b003 CR4: 0000000000370eb0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 ? die_addr+0x33/0x90
 ? exc_general_protection+0x1a2/0x390
 ? asm_exc_general_protection+0x22/0x30
 ? free_pcppages_bulk+0x12f/0x1e0
 ? free_pcppages_bulk+0x10d/0x1e0
 free_unref_page_commit+0x14d/0x2b0
 free_unref_page+0x18a/0x3e0
 skb_release_data+0x10d/0x180
 __kfree_skb+0x25/0x30
 tcp_ack+0x70d/0x14d0
 ? tcp_v6_rcv+0xf3c/0x1240
 tcp_rcv_established+0x5a9/0x760
 tcp_v6_do_rcv+0xd3/0x4a0
 tcp_v6_rcv+0xf3c/0x1240
 ? ip6_sublist_rcv+0x231/0x270
 ip6_protocol_deliver_rcu+0x56/0x450
 ip6_input+0xbf/0xe0
 ? tcp_v6_early_demux+0xb2/0x190
 ip6_sublist_rcv_finish+0x32/0x40
 ip6_sublist_rcv+0x231/0x270
 ? ip6_sublist_rcv+0x270/0x270
 ipv6_list_rcv+0xfc/0x120
 __netif_receive_skb_list_core+0x180/0x1e0
 netif_receive_skb_list_internal+0x1b5/0x2c0
 napi_complete_done+0x6f/0x190
 mlx5e_napi_poll+0x149/0x6a0 [mlx5_core]
 __napi_poll+0x24/0x190
 net_rx_action+0x328/0x3b0
 ? mlx5_eq_comp_int+0x1bc/0x1e0 [mlx5_core]
 ? notifier_call_chain+0x35/0xa0
 handle_softirqs+0xcc/0x270
 irq_exit_rcu+0x67/0x90
 common_interrupt+0x7f/0xa0
 </IRQ>
 <TASK>
 asm_common_interrupt+0x22/0x40
RIP: 0010:default_idle+0x13/0x20
Code: c0 08 00 00 00 4d 29 c8 4c 01 c7 4c 29 c2 e9 72 ff ff ff cc cc cc cc 8b 05 ca 29 4e 01 85 c0 7e 07 0f 00 2d f1 5a 25 00 fb f4 <fa> c3 66 66 2e 0f 1f 84 00 00 00 00 00 65 48 8b 35 38 7f 46 7e f0
RSP: 0018:ffff8881018cbee0 EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff88810189d500 RCX: 7fffffffffffffff
RDX: 0000000000000000 RSI: 000000089ca81700 RDI: 000000000014f654
RBP: 0000000000000004 R08: 7fffffffffffffff R09: 00000000fffeff0f
R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
 default_idle_call+0x39/0xd0
 do_idle+0x1ab/0x1c0
 cpu_startup_entry+0x25/0x30
 start_secondary+0x105/0x130
 common_startup_64+0x129/0x138
 </TASK>
Modules linked in: xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_nat br_netfilter rpcsec_gss_krb5 auth_rpcgss oid_registry overlay mlx5_ib zram zsmalloc mlx5_core rpcrdma rdma_ucm ib_uverbs ib_iser libiscsi scsi_transport_iscsi ib_umad rdma_cm ib_ipoib iw_cm ib_cm fuse ib_core
---[ end trace 0000000000000000 ]---
RIP: 0010:free_pcppages_bulk+0x12f/0x1e0
Code: 89 34 24 e8 a3 ed ff ff 49 8b 14 24 45 31 c9 4c 89 ff 49 89 c0 89 44 24 20 49 8b 44 24 08 8b 4c 24 0c 48 8b 34 24 48 89 42 08 <48> 89 10 48 8b 54 24 18 48 b8 00 01 00 00 00 00 ad de 49 89 04 24
RSP: 0018:ffff88885f905888 EFLAGS: 00010046
RAX: dead000000000122 RBX: ffff88885f932810 RCX: 0000000000000000
RDX: ffff88885f932830 RSI: 00000000001144a0 RDI: ffffea0004512800
RBP: 0000000000000001 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000001 R11: dead000000000100 R12: ffffea0004512808
R13: 000000000000003a R14: ffff88885f932800 R15: ffffea0004512800
FS:  0000000000000000(0000) GS:ffff88885f900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000056063c04b2e8 CR3: 000000000282b003 CR4: 0000000000370eb0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Kernel panic - not syncing: Fatal exception in interrupt
Shutting down cpus with NMI
Kernel Offset: disabled
---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---


Example #3:

BUG: kernel NULL pointer dereference, address: 0000000000000008
#PF: supervisor write access in kernel mode
#PF: error_code(0x0002) - not-present page
PGD 108898067 P4D 108898067 PUD 108891067 PMD 0
Oops: Oops: 0002 [#1] SMP
CPU: 1 PID: 1157 Comm: nginx_openssl_3 Not tainted 6.10.0-bisect+ #26
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
RIP: 0010:__page_cache_release+0xc7/0x260
Code: 8b 03 48 8b 53 08 48 c1 ed 12 83 e5 01 48 c1 e8 08 83 f5 01 83 e0 01 40 0f b6 ed 01 ed 3c 01 48 8b 43 10 83 dd ff 44 8d 7d 01 <48> 89 42 08 48 89 10 48 b8 00 01 00 00 00 00 ad de 48 89 43 08 48
RSP: 0018:ffff888110197b78 EFLAGS: 00010013
RAX: dead000000000122 RBX: ffffea0004e9aa00 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffff888110197bc8 RDI: ffff8881001e1050
RBP: 0000000000000002 R08: 000000000000005a R09: 00000000000009bc
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8881001e1000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000003
FS:  00007fa91da46740(0000) GS:ffff88885f840000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000008 CR3: 000000010889c001 CR4: 0000000000370eb0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 ? __die+0x20/0x60
 ? page_fault_oops+0x150/0x3e0
 ? exc_page_fault+0x74/0x130
 ? asm_exc_page_fault+0x22/0x30
 ? __page_cache_release+0xc7/0x260
 ? __page_cache_release+0x84/0x260
 ? folio_activate_fn+0x2d0/0x2d0
 folios_put_refs+0x6d/0x170
 filemap_splice_read+0x2b8/0x300
 ? tls_push_sg+0x13e/0x220
 ? tls_push_data+0x6bd/0xa40
 nfs_file_splice_read+0x78/0xa0
 splice_direct_to_actor+0xb0/0x230
 ? splice_file_range_actor+0x40/0x40
 do_splice_direct+0x73/0xb0
 ? propagate_umount+0x560/0x560
 do_sendfile+0x33b/0x3e0
 __x64_sys_sendfile64+0x5d/0xd0
 do_syscall_64+0x4c/0x100
 entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP: 0033:0x7fa91d905dae
Code: c3 0f 1f 00 4c 89 d2 4c 89 c6 e9 fd fd ff ff 0f 1f 44 00 00 31 c0 c3 0f 1f 44 00 00 f3 0f 1e fa 49 89 ca b8 28 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 4a 40 0f 00 f7 d8 64 89 01 48
RSP: 002b:00007ffda039ab98 EFLAGS: 00000202 ORIG_RAX: 0000000000000028
RAX: ffffffffffffffda RBX: 0000000029e45110 RCX: 00007fa91d905dae
RDX: 00007ffda039aba8 RSI: 0000000000000031 RDI: 000000000000001e
RBP: 0000000000000031 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000003e80 R11: 0000000000000202 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000003e80 R15: 0000000029e02c88
 </TASK>
Modules linked in: xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_nat br_netfilter rpcsec_gss_krb5 auth_rpcgss oid_registry overlay mlx5_ib zram zsmalloc mlx5_core rpcrdma rdma_ucm ib_uverbs ib_iser libiscsi scsi_transport_iscsi ib_umad rdma_cm ib_ipoib iw_cm ib_cm fuse ib_core
CR2: 0000000000000008
---[ end trace 0000000000000000 ]---
BUG: kernel NULL pointer dereference, address: 0000000000000008
RIP: 0010:__page_cache_release+0xc7/0x260
#PF: supervisor write access in kernel mode
Code: 8b 03 48 8b 53 08 48 c1 ed 12 83 e5 01 48 c1 e8 08 83 f5 01 83 e0 01 40 0f b6 ed 01 ed 3c 01 48 8b 43 10 83 dd ff 44 8d 7d 01 <48> 89 42 08 48 89 10 48 b8 00 01 00 00 00 00 ad de 48 89 43 08 48
#PF: error_code(0x0002) - not-present page
RSP: 0018:ffff888110197b78 EFLAGS: 00010013
PGD 1092fc067

P4D 1092fc067
RAX: dead000000000122 RBX: ffffea0004e9aa00 RCX: 0000000000000000
PUD 1092fb067 PMD 0
RDX: 0000000000000000 RSI: ffff888110197bc8 RDI: ffff8881001e1050

RBP: 0000000000000002 R08: 000000000000005a R09: 00000000000009bc
Oops: Oops: 0002 [#2] SMP
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8881001e1000
CPU: 3 PID: 1159 Comm: nginx_openssl_3 Tainted: G      D 6.10.0-bisect+ #26
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000003
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014
FS:  00007fa91da46740(0000) GS:ffff88885f840000(0000) knlGS:0000000000000000
RIP: 0010:__page_cache_release+0xc7/0x260
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Code: 8b 03 48 8b 53 08 48 c1 ed 12 83 e5 01 48 c1 e8 08 83 f5 01 83 e0 01 40 0f b6 ed 01 ed 3c 01 48 8b 43 10 83 dd ff 44 8d 7d 01 <48> 89 42 08 48 89 10 48 b8 00 01 00 00 00 00 ad de 48 89 43 08 48
CR2: 0000000000000008 CR3: 000000010889c001 CR4: 0000000000370eb0
RSP: 0018:ffff888124553cb8 EFLAGS: 00010013
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000

DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
RAX: dead000000000122 RBX: ffffea0004e9aa00 RCX: 0000000000000000
note: nginx_openssl_3[1157] exited with irqs disabled
RDX: 0000000000000000 RSI: ffff888124553d08 RDI: ffff888110672850
RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000001
R10: 0000000000000001 R11: 0000000000000000 R12: ffff888110672800
R13: 000000000000008e R14: 0000000000000058 R15: 0000000000000003
FS:  00007fa91da46740(0000) GS:ffff88885f8c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000008 CR3: 00000001092ff006 CR4: 0000000000370eb0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 ? __die+0x20/0x60
 ? page_fault_oops+0x150/0x3e0
 ? exc_page_fault+0x74/0x130
 ? asm_exc_page_fault+0x22/0x30
 ? __page_cache_release+0xc7/0x260
 ? __page_cache_release+0x84/0x260
 __folio_put+0x43/0xe0
 __filemap_get_folio+0x20c/0x2a0
 ext4_da_write_begin+0xe1/0x240
 generic_perform_write+0xe0/0x2c0
 ext4_buffered_write_iter+0x62/0xe0
 vfs_write+0x2c8/0x3f0
 ksys_write+0x5f/0xe0
 do_syscall_64+0x4c/0x100
 entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP: 0033:0x7fa91d9018b7
Code: 0f 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
RSP: 002b:00007ffda039af78 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 0000000029e45270 RCX: 00007fa91d9018b7
RDX: 0000000000000058 RSI: 0000000029e160f8 RDI: 0000000000000004
RBP: 0000000029ce3700 R08: 00000000cccccccd R09: 0000000000000000
R10: 0000000029e16142 R11: 0000000000000246 R12: 0000000000000058
R13: 0000000029ce35d0 R14: 0000000000000000 R15: 0000000029e45270
 </TASK>
Modules linked in: xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xt_addrtype iptable_nat nf_nat br_netfilter rpcsec_gss_krb5 auth_rpcgss oid_registry overlay mlx5_ib zram zsmalloc mlx5_core rpcrdma rdma_ucm ib_uverbs ib_iser libiscsi scsi_transport_iscsi ib_umad rdma_cm ib_ipoib iw_cm ib_cm fuse ib_core
CR2: 0000000000000008
---[ end trace 0000000000000000 ]---
RIP: 0010:__page_cache_release+0xc7/0x260
Code: 8b 03 48 8b 53 08 48 c1 ed 12 83 e5 01 48 c1 e8 08 83 f5 01 83 e0 01 40 0f b6 ed 01 ed 3c 01 48 8b 43 10 83 dd ff 44 8d 7d 01 <48> 89 42 08 48 89 10 48 b8 00 01 00 00 00 00 ad de 48 89 43 08 48
RSP: 0018:ffff888110197b78 EFLAGS: 00010013
RAX: dead000000000122 RBX: ffffea0004e9aa00 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffff888110197bc8 RDI: ffff8881001e1050
RBP: 0000000000000002 R08: 000000000000005a R09: 00000000000009bc





[Index of Archives]     [Linux Filesystem Development]     [Linux USB Development]     [Linux Media Development]     [Video for Linux]     [Linux NILFS]     [Linux Audio Users]     [Yosemite Info]     [Linux SCSI]

  Powered by Linux