Re: cifs

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Fix for the insmod/rmmod netfs bug

See attached


On Mon, May 13, 2024 at 2:34 AM Steve French <smfrench@xxxxxxxxx> wrote:
>
> The problem with the recent netfs/folio series is easy to repro, and
> doesn't show up if I remove the mempools patch:
>
> Author: David Howells <dhowells@xxxxxxxxxx>
> Date:   Fri Mar 15 18:03:30 2024 +0000
>
>     cifs: Add mempools for cifs_io_request and cifs_io_subrequest structs
>
>     Add mempools for the allocation of cifs_io_request and cifs_io_subrequest
>     structs for netfslib to use so that it can guarantee eventual allocation in
>     writeback.
>
> Repro is just to do modprobe and then rmmod
>
> [root@fedora29 xfstests-dev]# modprobe cifs
> [root@fedora29 xfstests-dev]# dmesg -c
> [  589.547809] Key type cifs.spnego registered
> [  589.547857] Key type cifs.idmap registered
> [root@fedora29 xfstests-dev]# rmmod cifs
> Segmentation fault
>
> [  593.793058] RIP: 0010:free_large_kmalloc+0x78/0xb0
> [  593.793063] Code: 74 0a 5d 41 5c 41 5d c3 cc cc cc cc 48 89 ef 5d
> 41 5c 41 5d e9 99 06 f4 ff 48 c7 c6 50 cf 38 9d 48 89 ef e8 7a f4 f8
> ff 0f 0b <0f> 0b 80 3d a6 3d 91 02 00 41 bc 00 f0 ff ff 75 a2 4c 89 ee
> 48 c7
> [  593.793068] RSP: 0018:ff1100011ceafe00 EFLAGS: 00010246
> [  593.793074] RAX: 0017ffffc0000000 RBX: 1fe22000239d5fc6 RCX: dffffc0000000000
> [  593.793078] RDX: ffd4000009265808 RSI: ffffffffc1960140 RDI: ffd4000009265800
> [  593.793082] RBP: ffd4000009265800 R08: ffffffff9b287a70 R09: 0000000000000001
> [  593.793086] R10: ffffffff9df472e7 R11: 0000000000000001 R12: ffffffffc195ff60
> [  593.793090] R13: ffffffffc1960140 R14: 0000000000000000 R15: 0000000000000000
> [  593.793093] FS:  00007fd5849cc280(0000) GS:ff110004cb200000(0000)
> knlGS:0000000000000000
> [  593.793098] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [  593.793101] CR2: 000055c6c44c7d58 CR3: 000000010da2a004 CR4: 0000000000371ef0
> [  593.793110] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [  593.793114] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [  593.793118] Call Trace:
> [  593.793121]  <TASK>
> [  593.793125]  ? __warn+0xa4/0x220
> [  593.793133]  ? free_large_kmalloc+0x78/0xb0
> [  593.793140]  ? report_bug+0x1d4/0x1e0
> [  593.793151]  ? handle_bug+0x42/0x80
> [  593.793158]  ? exc_invalid_op+0x18/0x50
> [  593.793164]  ? asm_exc_invalid_op+0x1a/0x20
> [  593.793178]  ? rcu_is_watching+0x20/0x50
> [  593.793188]  ? free_large_kmalloc+0x78/0xb0
> [  593.793197]  exit_cifs+0x89/0x6a0 [cifs]
> [  593.793363]  __do_sys_delete_module.constprop.0+0x23f/0x450
> [  593.793370]  ? __pfx___do_sys_delete_module.constprop.0+0x10/0x10
> [  593.793375]  ? mark_held_locks+0x24/0x90
> [  593.793383]  ? __x64_sys_close+0x54/0xa0
> [  593.793388]  ? lockdep_hardirqs_on_prepare+0x139/0x200
> [  593.793394]  ? kasan_quarantine_put+0x97/0x1f0
> [  593.793404]  ? mark_held_locks+0x24/0x90
> [  593.793414]  do_syscall_64+0x78/0x180
> [  593.793421]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
> [  593.793427] RIP: 0033:0x7fd584aecd4b
> [  593.793433] Code: 73 01 c3 48 8b 0d 3d 11 0c 00 f7 d8 64 89 01 48
> 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00
> 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 0d 11 0c 00 f7 d8 64 89
> 01 48
> [  593.793437] RSP: 002b:00007ffe0a36ec18 EFLAGS: 00000206 ORIG_RAX:
> 00000000000000b0
> [  593.793443] RAX: ffffffffffffffda RBX: 000055c6c44bd7a0 RCX: 00007fd584aecd4b
> [  593.793447] RDX: 000000000000000a RSI: 0000000000000800 RDI: 000055c6c44bd808
> [  593.793451] RBP: 0000000000000000 R08: 00007ffe0a36db91 R09: 0000000000000000
> [  593.793454] R10: 00007fd584b5eae0 R11: 0000000000000206 R12: 00007ffe0a36ee40
> [  593.793458] R13: 00007ffe0a3706d1 R14: 000055c6c44bd260 R15: 000055c6c44bd7a0
> [  593.793474]  </TASK>
> [  593.793477] irq event stamp: 12729
> [  593.793480] hardirqs last  enabled at (12735): [<ffffffff9b25d2eb>]
> console_unlock+0x15b/0x170
> [  593.793487] hardirqs last disabled at (12740): [<ffffffff9b25d2d0>]
> console_unlock+0x140/0x170
> [  593.793492] softirqs last  enabled at (11910): [<ffffffff9b16499e>]
> __irq_exit_rcu+0xfe/0x120
> [  593.793498] softirqs last disabled at (11901): [<ffffffff9b16499e>]
> __irq_exit_rcu+0xfe/0x120
> [  593.793503] ---[ end trace 0000000000000000 ]---
> [  593.793546] object pointer: 0x00000000da6e868b
> [  593.793550] ==================================================================
> [  593.793553] BUG: KASAN: invalid-free in exit_cifs+0x89/0x6a0 [cifs]
> [  593.793698] Free of addr ffffffffc1960140 by task rmmod/1306
>
> [  593.793703] CPU: 4 PID: 1306 Comm: rmmod Tainted: G        W
>   6.9.0 #1
> [  593.793707] Hardware name: Red Hat KVM, BIOS 1.16.1-1.el9 04/01/2014
> [  593.793709] Call Trace:
> [  593.793711]  <TASK>
> [  593.793714]  dump_stack_lvl+0x79/0xb0
> [  593.793718]  print_report+0xcb/0x620
> [  593.793724]  ? exit_cifs+0x89/0x6a0 [cifs]
> [  593.793861]  ? exit_cifs+0x89/0x6a0 [cifs]
> [  593.794002]  kasan_report_invalid_free+0x9a/0xc0
> [  593.794008]  ? exit_cifs+0x89/0x6a0 [cifs]
> [  593.794173]  free_large_kmalloc+0x38/0xb0
> [  593.794178]  exit_cifs+0x89/0x6a0 [cifs]
> [  593.794327]  __do_sys_delete_module.constprop.0+0x23f/0x450
> [  593.794331]  ? __pfx___do_sys_delete_module.constprop.0+0x10/0x10
> [  593.794335]  ? mark_held_locks+0x24/0x90
> [  593.794339]  ? __x64_sys_close+0x54/0xa0
> [  593.794342]  ? lockdep_hardirqs_on_prepare+0x139/0x200
> [  593.794347]  ? kasan_quarantine_put+0x97/0x1f0
> [  593.794352]  ? mark_held_locks+0x24/0x90
> [  593.794357]  do_syscall_64+0x78/0x180
> [  593.794361]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
> [  593.794367] RIP: 0033:0x7fd584aecd4b
> [  593.794370] Code: 73 01 c3 48 8b 0d 3d 11 0c 00 f7 d8 64 89 01 48
> 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00
> 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 0d 11 0c 00 f7 d8 64 89
> 01 48
> [  593.794373] RSP: 002b:00007ffe0a36ec18 EFLAGS: 00000206 ORIG_RAX:
> 00000000000000b0
> [  593.794377] RAX: ffffffffffffffda RBX: 000055c6c44bd7a0 RCX: 00007fd584aecd4b
> [  593.794380] RDX: 000000000000000a RSI: 0000000000000800 RDI: 000055c6c44bd808
> [  593.794382] RBP: 0000000000000000 R08: 00007ffe0a36db91 R09: 0000000000000000
> [  593.794385] R10: 00007fd584b5eae0 R11: 0000000000000206 R12: 00007ffe0a36ee40
> [  593.794387] R13: 00007ffe0a3706d1 R14: 000055c6c44bd260 R15: 000055c6c44bd7a0
> [  593.794394]  </TASK>
>
> [  593.794398] The buggy address belongs to the variable:
> [  593.794399]  cifs_io_subrequest_pool+0x0/0xfffffffffff3dec0 [cifs]
>
> [  593.794557] Memory state around the buggy address:
> [  593.794559]  ffffffffc1960000: 00 00 f9 f9 f9 f9 f9 f9 00 00 f9 f9
> f9 f9 f9 f9
> [  593.794562]  ffffffffc1960080: 00 00 f9 f9 f9 f9 f9 f9 00 00 f9 f9
> f9 f9 f9 f9
> [  593.794565] >ffffffffc1960100: 00 00 f9 f9 f9 f9 f9 f9 00 00 00 00
> 00 00 00 00
> [  593.794567]                                            ^
> [  593.794570]  ffffffffc1960180: 00 00 00 00 00 00 00 00 00 00 00 00
> 00 00 00 f9
> [  593.794572]  ffffffffc1960200: f9 f9 f9 f9 00 00 00 00 00 00 00 00
> 00 00 00 00
> [  593.794575] ==================================================================
>
> On Sat, May 11, 2024 at 12:59 PM Steve French <smfrench@xxxxxxxxx> wrote:
> >
> > This was running against linux-next as of about an hour ago
> >
> > On Sat, May 11, 2024 at 12:53 PM Steve French <smfrench@xxxxxxxxx> wrote:
> > >
> > > Tried running the regression tests against for-next and saw crash
> > > early in the test run in
> > >
> > > # FS QA Test No. cifs/006
> > > #
> > > # check deferred closes on handles of deleted files
> > > #
> > > umount: /mnt/test: not mounted.
> > > umount: /mnt/test: not mounted.
> > > umount: /mnt/scratch: not mounted.
> > > umount: /mnt/scratch: not mounted.
> > > ./run-xfstests.sh: line 25: 4556 Segmentation fault rmmod cifs
> > > modprobe: ERROR: could not insert 'cifs': Device or resource busy
> > >
> > > More information here:
> > > http://smb311-linux-testing.southcentralus.cloudapp.azure.com/#/builders/5/builds/123/steps/14/logs/stdio
> > >
> > > Are you also seeing that?  There are not many likely candidates for
> > > what patch is causing the problem (could be related to the folios
> > > changes) e.g.
> > >
> > > 7c1ac89480e8 cifs: Enable large folio support
> > > 3ee1a1fc3981 cifs: Cut over to using netfslib
> > > 69c3c023af25 cifs: Implement netfslib hooks
> > > c20c0d7325ab cifs: Make add_credits_and_wake_if() clear deducted credits
> > > edea94a69730 cifs: Add mempools for cifs_io_request and
> > > cifs_io_subrequest structs
> > > 3758c485f6c9 cifs: Set zero_point in the copy_file_range() and
> > > remap_file_range()
> > > 1a5b4edd97ce cifs: Move cifs_loose_read_iter() and
> > > cifs_file_write_iter() to file.c
> > > dc5939de82f1 cifs: Replace the writedata replay bool with a netfs sreq flag
> > > 56257334e8e0 cifs: Make wait_mtu_credits take size_t args
> > > ab58fbdeebc7 cifs: Use more fields from netfs_io_subrequest
> > > a975a2f22cdc cifs: Replace cifs_writedata with a wrapper around
> > > netfs_io_subrequest
> > > 753b67eb630d cifs: Replace cifs_readdata with a wrapper around
> > > netfs_io_subrequest
> > > 0f7c0f3f5150 cifs: Use alternative invalidation to using launder_folio
> > > 2e9d7e4b984a mm: Remove the PG_fscache alias for PG_private_2
> > >
> > > Any ideas?
> > >
> > > --
> > > Thanks,
> > >
> > > Steve
> >
> >
> >
> > --
> > Thanks,
> >
> > Steve
>
>
>
> --
> Thanks,
>
> Steve



--
Thanks,

Steve
From ea19a7a48edb2de8ebeef509fa16557394d5d3c4 Mon Sep 17 00:00:00 2001
From: Steve French <stfrench@xxxxxxxxxxxxx>
Date: Mon, 13 May 2024 12:49:22 -0500
Subject: [PATCH] cifs: Change from mempool_destroy to mempool_exit for request
 pools

insmod followed by rmmod was oopsing with the new mempools cifs request patch

Fixes: edea94a69730 ("cifs: Add mempools for cifs_io_request and cifs_io_subrequest structs")
Suggested-by: David Howells <dhowells@xxxxxxxxxx>
Reviewed-by: Enzo Matsumiya <ematsumiya@xxxxxxx>
Signed-off-by: Steve French <stfrench@xxxxxxxxxxxxx>
---
 fs/smb/client/cifsfs.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/smb/client/cifsfs.c b/fs/smb/client/cifsfs.c
index 763d17870e0b..c8b52ac94ce2 100644
--- a/fs/smb/client/cifsfs.c
+++ b/fs/smb/client/cifsfs.c
@@ -1790,9 +1790,9 @@ static int cifs_init_netfs(void)
 
 static void cifs_destroy_netfs(void)
 {
-	mempool_destroy(&cifs_io_subrequest_pool);
+	mempool_exit(&cifs_io_subrequest_pool);
 	kmem_cache_destroy(cifs_io_subrequest_cachep);
-	mempool_destroy(&cifs_io_request_pool);
+	mempool_exit(&cifs_io_request_pool);
 	kmem_cache_destroy(cifs_io_request_cachep);
 }
 
-- 
2.40.1


[Index of Archives]     [Linux Ext4 Filesystem]     [Union Filesystem]     [Filesystem Testing]     [Ceph Users]     [Ecryptfs]     [NTFS 3]     [AutoFS]     [Kernel Newbies]     [Share Photos]     [Security]     [Netfilter]     [Bugtraq]     [Yosemite News]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux Cachefs]     [Reiser Filesystem]     [Linux RAID]     [NTFS 3]     [Samba]     [Device Mapper]     [CEPH Development]

  Powered by Linux