On Tue, Jul 13, 2021 at 06:53:04PM +0900, William Breathitt Gray wrote: > Suppose I open the chrdev from a userspace application and keep it open, > but then remove the respective driver and Counter subsystem module from > my system. The devm_counter_release() and counter_exit() functions will > be called as expected; the counter_chrdev_release() function will not be > called yet, but that is expected because the chrdev is still open by > userspace. If I try to break out of my userspace application, I expect > counter_chrdev_release() to finally be called, but this does not happen. > Instead, my userspace application stalls and I see the following error > in my dmesg: > > [ 172.859570] BUG: unable to handle page fault for address: ffffffffc09ae298 > [ 172.859594] #PF: supervisor read access in kernel mode > [ 172.859598] #PF: error_code(0x0000) - not-present page > [ 172.859603] PGD 23615067 P4D 23615067 PUD 23617067 PMD 1029ad067 PTE 0 > [ 172.859623] Oops: 0000 [#1] SMP NOPTI > [ 172.859629] CPU: 2 PID: 2485 Comm: counter_example Not tainted 5.13.0+ #1 > [ 172.859640] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS d55cb5a 04/01/2014 > [ 172.859645] RIP: 0010:filp_close+0x29/0x70 > [ 172.859662] Code: 90 0f 1f 44 00 00 55 48 89 e5 41 56 41 55 41 54 48 8b 47 38 48 85 c0 0f 84 b7 40 86 00 48 8b 47 28 49 89 fc 49 89 f5 45 31 f6 <48> 8b 40 78 48 85 c0 74 08 ff d0 0f 1f 00 41 89 c6 41 f6 44 24 45 > [ 172.859669] RSP: 0018:ffffad31c0ee7cb0 EFLAGS: 00010246 > [ 172.859675] RAX: ffffffffc09ae220 RBX: 0000000000000001 RCX: 0000000000000001 > [ 172.859680] RDX: ffff9a43829708e0 RSI: ffff9a4382970840 RDI: ffff9a43821f4f00 > [ 172.859684] RBP: ffffad31c0ee7cc8 R08: 0000000000000001 R09: 0000000000000001 > [ 172.859687] R10: ffffffffffff4d00 R11: ffff9a43933c6e10 R12: ffff9a43821f4f00 > [ 172.859691] R13: ffff9a4382970840 R14: 0000000000000000 R15: 0000000000000003 > [ 172.859694] FS: 0000000000000000(0000) GS:ffff9a44b7d00000(0000) knlGS:0000000000000000 > [ 172.859699] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 172.859704] CR2: ffffffffc09ae298 CR3: 0000000023610001 CR4: 0000000000370ee0 > [ 172.859713] Call Trace: > [ 172.859730] put_files_struct+0x73/0xd0 > [ 172.859738] exit_files+0x49/0x50 > [ 172.859743] do_exit+0x33b/0xa20 > [ 172.859751] do_group_exit+0x3b/0xb0 > [ 172.859758] get_signal+0x16f/0x8b0 > [ 172.859766] ? _copy_to_user+0x20/0x30 > [ 172.859774] ? put_timespec64+0x3d/0x60 > [ 172.859784] arch_do_signal_or_restart+0xf3/0x850 > [ 172.859794] ? hrtimer_nanosleep+0x9f/0x120 > [ 172.859802] ? __hrtimer_init+0xd0/0xd0 > [ 172.859808] exit_to_user_mode_prepare+0x122/0x1b0 > [ 172.859816] syscall_exit_to_user_mode+0x27/0x50 > [ 172.859825] do_syscall_64+0x48/0xc0 > [ 172.859831] entry_SYSCALL_64_after_hwframe+0x44/0xae > [ 172.859839] RIP: 0033:0x7f07f8b9951a > [ 172.859850] Code: Unable to access opcode bytes at RIP 0x7f07f8b994f0. > [ 172.859853] RSP: 002b:00007ffc0d12c230 EFLAGS: 00000246 ORIG_RAX: 00000000000000e6 > [ 172.859860] RAX: fffffffffffffdfc RBX: ffffffffffffff01 RCX: 00007f07f8b9951a > [ 172.859863] RDX: 00007ffc0d12c2b0 RSI: 0000000000000000 RDI: 0000000000000000 > [ 172.859867] RBP: 0000000000000000 R08: 0000000000000000 R09: 00007ffc0d12c1c6 > [ 172.859871] R10: 00007ffc0d12c2b0 R11: 0000000000000246 R12: 00007ffc0d12c2b0 > [ 172.859874] R13: 00007ffc0d12c2b0 R14: 0000000000000000 R15: 0000000000000000 > [ 172.859886] Modules linked in: intel_rapl_msr intel_rapl_common kvm_intel kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel nls_iso8859_1 aesni_intel crypto_simd cryptd rapl drm_ttm_helper ttm uvcvideo drm_kms_helper videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_common input_leds syscopyarea videodev sysfillrect sysimgblt fb_sys_fops cec rc_core joydev mc drm serio_raw mac_hid qemu_fw_cfg sch_fq_codel msr parport_pc ppdev lp parport virtio_rng ip_tables x_tables autofs4 hid_generic usbhid hid virtio_net psmouse net_failover i2c_piix4 virtio_blk failover pata_acpi floppy [last unloaded: counter] > [ 172.859995] CR2: ffffffffc09ae298 > [ 172.860009] ---[ end trace e7d3d7da1a73b8f4 ]--- > [ 172.860013] RIP: 0010:filp_close+0x29/0x70 > [ 172.860021] Code: 90 0f 1f 44 00 00 55 48 89 e5 41 56 41 55 41 54 48 8b 47 38 48 85 c0 0f 84 b7 40 86 00 48 8b 47 28 49 89 fc 49 89 f5 45 31 f6 <48> 8b 40 78 48 85 c0 74 08 ff d0 0f 1f 00 41 89 c6 41 f6 44 24 45 > [ 172.860027] RSP: 0018:ffffad31c0ee7cb0 EFLAGS: 00010246 > [ 172.860031] RAX: ffffffffc09ae220 RBX: 0000000000000001 RCX: 0000000000000001 > [ 172.860034] RDX: ffff9a43829708e0 RSI: ffff9a4382970840 RDI: ffff9a43821f4f00 > [ 172.860038] RBP: ffffad31c0ee7cc8 R08: 0000000000000001 R09: 0000000000000001 > [ 172.860041] R10: ffffffffffff4d00 R11: ffff9a43933c6e10 R12: ffff9a43821f4f00 > [ 172.860044] R13: ffff9a4382970840 R14: 0000000000000000 R15: 0000000000000003 > [ 172.860047] FS: 0000000000000000(0000) GS:ffff9a44b7d00000(0000) knlGS:0000000000000000 > [ 172.860052] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 172.860056] CR2: ffffffffc09ae298 CR3: 0000000023610001 CR4: 0000000000370ee0 > [ 172.860073] Fixing recursive fault but reboot is needed! > > It looks like faults in filp_close() before counter_chrdev_release() is > called. Is this issue manifesting because counter_exit() was called > earlier while the chrdev was still open? After giving this some more thought I realized that once the counter module is unloaded, the counter_fops callbacks are lost and thus the fops for the chrdev are no longer be valid. This means that counter_chrdev_release will need to be called before the counter module is unloaded. How can I guarantee this if a userspace application may still have the chrdev indefinitely open? In counter_chrdev_remove(), should I walk through the open chrdevs and release each one? William Breathitt Gray
Attachment:
signature.asc
Description: PGP signature
