Hello, I'm including some kernel stack traces below. They were spurious, so I hesitate to file bug. I have also filed some similar bugs before, which were back then more sure to occur. I also see many recent posts mentioning similar kernel stack traces and related to ntfs3 and folios. Are there any bugs fixes/issue-nos related to this that are being addressed, already committed to kernel-6.1, 6.2, 6.3 that I should be aware of? thx -Gana Host: fedora-37/kernel-6.0.15-300.fc37.x86_64 qemu: 7.2.0 Guest: fedora-37/kernel-6.0.15-300.fc37.x86_64 Prior similar bugs: Bug 215460 fs/ntfs3: page_cache_Ra_unbounded on rsync from ntfs3 to ext4 https://bugzilla.kernel.org/show_bug.cgi?id=215460 Bug 215563 during rsync, vm guest kernel trace arising from memcg_kmem_charge_page alloca_pages https://bugzilla.kernel.org/show_bug.cgi?id=215563 Essentially, I do the same standard steps as mentioned in those bugs. The bug I am really stress testing for is https://gitlab.com/qemu-project/qemu/-/issues/727 In it, two vdisks are connected to a qemu vm. The destination vdisk starts of as an empty dynamic vdisk, the source vdisk contains synthetic generated data The empty dynamic vdisk is inflated from a tiny size to 85Gib in under 16 minutes. reproducibility: Spurious 2 out of 6 testruns went to completion. Steps to reproduce: same as before On a qcow2-dynamic vdisk, that starts of empty, stress test copy about 85Gib of data to inflate its size. src: ntfs3/sdc/vhdx/ntfs3/hdd dst: ntfs3/sdb/qcow2-dynamic/ntfs3/ssd Below are 6 testruns with 2 strack-traces. Prior to these I have tested the destination vdisk being 93Gib-raw-img-fixed, fixed-93Gib-qemu-fixed and 93Gib-vhdx-fixed. These did not give any problem. The write stress on the expanding qcow2-dynamic vdisk is required. RUN1: guest: [root@sirius ~]# ls -l /mnt/a17/gkpics01.qcow2 -rw-r--r--. 1 gana gana 198096 Jan 3 15:27 /mnt/a17/gkpics01.qcow2 [root@sirius ~]# ls -l /mnt/a17/gkpics01.qcow2 # after gdisk -rw-r--r--. 1 gana gana 524288 Jan 3 15:34 /mnt/a17/gkpics01.qcow2 [root@sirius ~]# run the rsync step : : photos001/D0038_16ab9e88/File_0583.bin Tue Jan 3 03:42:36 PM IST 2023 rsync: connection unexpectedly closed (635379 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(231) [generator=3.2.7] Tue Jan 3 03:57:57 PM IST 2023 host: [root@sirius ~]# ls -l /mnt/a17/gkpics01.qcow2 -rw-r--r--. 1 gana gana 90386464768 Jan 3 15:28 /mnt/a17/gkpics01.qcow2 : [ 995.145136] BUG: unable to handle page fault for address: 00000000486501e3 [ 995.145162] #PF: supervisor write access in kernel mode [ 995.145167] #PF: error_code(0x0002) - not-present page [ 995.145172] PGD 0 P4D 0 [ 995.145188] Oops: 0002 [#1] PREEMPT SMP PTI [ 995.145199] CPU: 7 PID: 2531 Comm: rsync Not tainted 6.0.15-300.fc37.x86_64 #1 [ 995.145211] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS edk2-20221117gitfff6d81270b5-8.fc37 unknown [ 995.145216] RIP: 0010:io_schedule+0x41/0x70 [ 995.145231] Code: 8b ba 28 0d 00 00 89 c3 83 c8 04 c0 eb 02 88 82 90 09 00 00 83 e3 01 48 85 ff 74 0a be 01 00 00 00 e8 e3 58 8b ff e8 be fe ff <ff> 83 e3 01 65 48 8b 14 25 c0 fb 01 00 0f b6 82 90 09 00 00 c1 e3 [ 995.145237] RSP: 0018:ffffb70c81343b98 EFLAGS: 00010246 [ 995.145243] RAX: 0000000000400000 RBX: 0000000000000000 RCX: 0000000000000000 [ 995.145248] RDX: 0000000000000001 RSI: ffffffffac7aac9a RDI: 00000000ffffffff [ 995.145253] RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 [ 995.145256] R10: 0000000000000001 R11: 0000000000000007 R12: 0000000000000102 [ 995.145260] R13: ffffffffad008f18 R14: ffffb70c81343c00 R15: fffffb1d808c62c0 [ 995.145265] FS: 00007fba69929c40(0000) GS:ffff9b2b3bdc0000(0000) knlGS:0000000000000000 [ 995.145271] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 995.145276] CR2: 00000000486501e3 CR3: 0000000176606000 CR4: 00000000000006e0 [ 995.145285] Call Trace: [ 995.145290] <TASK> [ 995.145296] folio_wait_bit_common+0x12d/0x3a0 [ 995.145311] ? filemap_alloc_folio+0xc0/0xc0 [ 995.145322] filemap_get_pages+0x5f7/0x670 [ 995.145333] ? copy_page_to_iter+0x8a/0x2e0 [ 995.145342] ? workingset_activation+0xb1/0x1a0 [ 995.145352] filemap_read+0xbe/0x3d0 [ 995.145365] vfs_read+0x21a/0x2f0 [ 995.145374] ksys_read+0x5b/0xd0 [ 995.145380] do_syscall_64+0x5b/0x80 [ 995.145389] ? ksys_write+0x97/0xd0 [ 995.145392] ? syscall_exit_to_user_mode+0x17/0x40 [ 995.145397] ? do_syscall_64+0x67/0x80 [ 995.145402] ? do_syscall_64+0x67/0x80 [ 995.145407] ? syscall_exit_to_user_mode+0x17/0x40 [ 995.145411] ? do_syscall_64+0x67/0x80 [ 995.145416] entry_SYSCALL_64_after_hwframe+0x63/0xcd [ 995.145423] RIP: 0033:0x7fba6931e021 [ 995.148377] Code: 31 c0 e9 b2 fe ff ff 50 48 8d 3d 32 35 0a 00 e8 35 08 02 00 0f 1f 44 00 00 f3 0f 1e fa 80 3d dd 05 0e 00 00 74 13 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 57 c3 66 0f 1f 44 00 00 48 83 ec 28 48 89 54 [ 995.148384] RSP: 002b:00007fff3a2ff5f8 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 [ 995.148392] RAX: ffffffffffffffda RBX: 000055f6ad9b9a10 RCX: 00007fba6931e021 [ 995.148396] RDX: 000000000003c5ff RSI: 000055f6ada138c0 RDI: 0000000000000003 [ 995.148399] RBP: 000000000003c5ff R08: 000000000003c5ff R09: 000000000003c5ff [ 995.148403] R10: 000000000000000f R11: 0000000000000246 R12: 0000000000000000 [ 995.148406] R13: 0000000000000000 R14: 0000000000000000 R15: 000000000003c5ff [ 995.148417] </TASK> [ 995.148427] Modules linked in: ntfs3 uinput snd_seq_dummy snd_hrtimer snd_seq snd_seq_device snd_timer snd soundcore nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 rfkill ip_set nf_tables nfnetlink qrtr vfat fat ppdev iTCO_wdt intel_pmc_bxt iTCO_vendor_support joydev parport_pc parport i2c_i801 pcspkr i2c_smbus lpc_ich zram virtio_net net_failover virtio_console virtio_gpu failover serio_raw virtio_dma_buf aacraid mvsas libsas vmw_pvscsi megaraid_sas megaraid smartpqi pmcraid uas vhci_hcd usbip_core xhci_plat_hcd usb_storage nvme nvme_core nvme_common mptspi scsi_transport_spi mptsas mptscsih mptbase scsi_transport_sas sunrpc be2iscsi bnx2i cnic uio cxgb4i cxgb4 tls cxgb3i cxgb3 mdio libcxgbi libcxgb qla4xxx iscsi_boot_sysfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip6_tables ip_tables fuse [ 995.149789] qemu_fw_cfg [ 995.149807] CR2: 00000000486501e3 [ 995.149883] ---[ end trace 0000000000000000 ]--- [ 995.149889] RIP: 0010:io_schedule+0x41/0x70 [ 995.149898] Code: 8b ba 28 0d 00 00 89 c3 83 c8 04 c0 eb 02 88 82 90 09 00 00 83 e3 01 48 85 ff 74 0a be 01 00 00 00 e8 e3 58 8b ff e8 be fe ff <ff> 83 e3 01 65 48 8b 14 25 c0 fb 01 00 0f b6 82 90 09 00 00 c1 e3 [ 995.149902] RSP: 0018:ffffb70c81343b98 EFLAGS: 00010246 [ 995.149906] RAX: 0000000000400000 RBX: 0000000000000000 RCX: 0000000000000000 [ 995.149909] RDX: 0000000000000001 RSI: ffffffffac7aac9a RDI: 00000000ffffffff [ 995.149912] RBP: 0000000000000002 R08: 0000000000000000 R09: 0000000000000000 [ 995.149914] R10: 0000000000000001 R11: 0000000000000007 R12: 0000000000000102 [ 995.149916] R13: ffffffffad008f18 R14: ffffb70c81343c00 R15: fffffb1d808c62c0 [ 995.149918] FS: 00007fba69929c40(0000) GS:ffff9b2b3bdc0000(0000) knlGS:0000000000000000 [ 995.149920] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 995.149922] CR2: 00000000486501e3 CR3: 0000000176606000 CR4: 00000000000006e0 RUN2: system freeze RUN3: sort of freeze, gui responsive, but can't start new terminal/shell/new process RUN4: traceback but operable [ 131.484082] BUG: unable to handle page fault for address: 0000000000001001 [ 131.484091] #PF: supervisor write access in kernel mode [ 131.484094] #PF: error_code(0x0002) - not-present page [ 131.484096] PGD 0 P4D 0 [ 131.484104] Oops: 0002 [#1] PREEMPT SMP PTI [ 131.484110] CPU: 0 PID: 2558 Comm: rsync Not tainted 6.0.15-300.fc37.x86_64 #1 [ 131.484115] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS edk2-20221117gitfff6d81270b5-8.fc37 unknown [ 131.484117] RIP: 0010:copy_page_from_iter+0x76/0x170 [ 131.484123] Code: 4c 89 e2 4c 29 fe 4c 39 f6 49 0f 47 f6 48 2b 3d 40 ab 18 01 48 c1 ff 06 48 c1 e7 0c 48 03 3d 41 ab 18 01 4c 01 ff e8 09 fa ff <ff> 48 01 c5 49 29 c6 74 5f 48 85 c0 74 5a 49 01 c7 49 81 ff 00 10 [ 131.484125] RSP: 0018:ffffb70000c97bc8 EFLAGS: 00010286 [ 131.484127] RAX: 0000000000001000 RBX: ffffe5cfc0dc30c0 RCX: 0000000000004008 [ 131.484129] RDX: 0000000000004000 RSI: 00005616cc757138 RDI: ffff99ed770c4000 [ 131.484130] RBP: 0000000000003000 R08: 326e73452c4e6cb7 R09: 7227a8304c2683b2 [ 131.484131] R10: cb3f3c018be49eba R11: 219097ff2b911ec9 R12: ffffb70000c97d18 [ 131.484132] R13: 0000000000001000 R14: 0000000000005000 R15: 0000000000000000 [ 131.484134] FS: 00007f1634972c40(0000) GS:ffff99eebbc00000(0000) knlGS:0000000000000000 [ 131.484135] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 131.484137] CR2: 0000000000001001 CR3: 0000000019ec2000 CR4: 00000000000006f0 [ 131.484141] Call Trace: [ 131.484143] <TASK> [ 131.484147] skb_copy_datagram_from_iter+0x10e/0x1e0 [ 131.484153] unix_stream_sendmsg+0x29d/0x700 [ 131.484158] sock_sendmsg+0x5f/0x70 [ 131.484161] sock_write_iter+0x89/0xe0 [ 131.484164] vfs_write+0x34e/0x3e0 [ 131.484172] ksys_write+0x97/0xd0 [ 131.484174] do_syscall_64+0x5b/0x80 [ 131.484179] ? do_pselect.constprop.0+0xe9/0x150 [ 131.484184] ? syscall_exit_to_user_mode+0x17/0x40 [ 131.484187] ? do_syscall_64+0x67/0x80 [ 131.484189] ? exit_to_user_mode_prepare+0x18f/0x1f0 [ 131.484196] ? syscall_exit_to_user_mode+0x17/0x40 [ 131.484198] ? do_syscall_64+0x67/0x80 [ 131.484200] ? do_syscall_64+0x67/0x80 [ 131.484203] ? do_syscall_64+0x67/0x80 [ 131.484206] entry_SYSCALL_64_after_hwframe+0x63/0xcd [ 131.484209] RIP: 0033:0x7f163431e0c4 [ 131.484233] Code: 15 71 7d 0d 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 80 3d 3d 05 0e 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 48 83 ec 28 48 89 54 24 18 48 [ 131.484234] RSP: 002b:00007fff4a53a0f8 EFLAGS: 00000202 ORIG_RAX: 0000000000000001 [ 131.484237] RAX: ffffffffffffffda RBX: 0000000000000005 RCX: 00007f163431e0c4 [ 131.484238] RDX: 0000000000008008 RSI: 00005616cc753130 RDI: 0000000000000004 [ 131.484239] RBP: 00005616ca8dc080 R08: 0000000000008008 R09: 0000000000000000 [ 131.484248] R10: 00007fff4a53a1d0 R11: 0000000000000202 R12: 00007fff4a53a150 [ 131.484250] R13: 0000000000000002 R14: 00007fff4a53a250 R15: 00007fff4a53a1d0 [ 131.484254] </TASK> [ 131.484267] Modules linked in: ntfs3 uinput snd_seq_dummy snd_hrtimer snd_seq snd_seq_device snd_timer snd soundcore nft_objref nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set rfkill nf_tables nfnetlink qrtr vfat fat iTCO_wdt ppdev intel_pmc_bxt iTCO_vendor_support parport_pc i2c_i801 parport pcspkr i2c_smbus joydev lpc_ich zram virtio_net virtio_gpu net_failover serio_raw failover virtio_console virtio_dma_buf aacraid mvsas libsas vmw_pvscsi megaraid_sas megaraid smartpqi pmcraid uas vhci_hcd usbip_core xhci_plat_hcd usb_storage nvme nvme_core nvme_common mptspi scsi_transport_spi mptsas mptscsih mptbase scsi_transport_sas sunrpc be2iscsi bnx2i cnic uio cxgb4i cxgb4 tls cxgb3i cxgb3 mdio libcxgbi libcxgb qla4xxx iscsi_boot_sysfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip6_tables ip_tables fuse [ 131.484334] qemu_fw_cfg [ 131.484342] CR2: 0000000000001001 [ 131.484347] ---[ end trace 0000000000000000 ]--- [ 131.484349] RIP: 0010:copy_page_from_iter+0x76/0x170 [ 131.484353] Code: 4c 89 e2 4c 29 fe 4c 39 f6 49 0f 47 f6 48 2b 3d 40 ab 18 01 48 c1 ff 06 48 c1 e7 0c 48 03 3d 41 ab 18 01 4c 01 ff e8 09 fa ff <ff> 48 01 c5 49 29 c6 74 5f 48 85 c0 74 5a 49 01 c7 49 81 ff 00 10 [ 131.484355] RSP: 0018:ffffb70000c97bc8 EFLAGS: 00010286 [ 131.484357] RAX: 0000000000001000 RBX: ffffe5cfc0dc30c0 RCX: 0000000000004008 [ 131.484358] RDX: 0000000000004000 RSI: 00005616cc757138 RDI: ffff99ed770c4000 [ 131.484359] RBP: 0000000000003000 R08: 326e73452c4e6cb7 R09: 7227a8304c2683b2 [ 131.484360] R10: cb3f3c018be49eba R11: 219097ff2b911ec9 R12: ffffb70000c97d18 [ 131.484361] R13: 0000000000001000 R14: 0000000000005000 R15: 0000000000000000 [ 131.484363] FS: 00007f1634972c40(0000) GS:ffff99eebbc00000(0000) knlGS:0000000000000000 [ 131.484364] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 131.484365] CR2: 0000000000001001 CR3: 0000000019ec2000 CR4: 00000000000006f0 RUN5: success copy: Tue Jan 3 07:16:29 PM IST 2023 Tue Jan 3 07:32:07 PM IST 2023 16 min verify: Tue Jan 3 07:32:53 PM IST 2023 Tue Jan 3 07:48:19 PM IST 2023 Tue Jan 3 07:57:29 PM IST 2023 Tue Jan 3 08:13:25 PM IST 2023 16 min [root@sirius ~]# ls -l /mnt/a17/gkpics01.qcow2 ; df | grep sda17 -rw-r--r--. 1 gana gana 91077017600 Jan 3 19:56 /mnt/a17/gkpics01.qcow2 /dev/sda17 97517564 89051588 8465976 92% /mnt/a17 RUN6: success mounted src vdisk using ntfs-3g fuseblk driver. src: ntfs3g-fuseblk/sdc/vhdx/ntfs3/hdd dst: ntfs3/sdb/qcow2-dynamic/ntfs3/ssd copy Tue Jan 3 09:12:17 PM IST 2023 Tue Jan 3 09:28:51 PM IST 2023 16 min verify Tue Jan 3 09:30:23 PM IST 2023 Tue Jan 3 09:51:04 PM IST 2023 Tue Jan 3 10:14:27 PM IST 2023 Tue Jan 3 10:29:20 PM IST 2023 15 min
