On 02/18/2016 06:58 PM, Dave Chinner wrote: > On Tue, Feb 16, 2016 at 08:31:18PM -0500, Waiman Long wrote: >> This patch is a replacement of my previous list batching patch - >> https://lwn.net/Articles/674105/. Compared with the previous patch, >> this one provides better performance and fairness. However, it also >> requires a bit more changes in the VFS layer. >> >> This patchset is a derivative of Andi Kleen's patch on "Initial per >> cpu list for the per sb inode list" >> >> https://git.kernel.org/cgit/linux/kernel/git/ak/linux-misc.git/commit/?h=hle315/combined&id=f1cf9e715a40f44086662ae3b29f123cf059cbf4 >> >> Patch 1 introduces the per-cpu list. >> >> Patch 2 modifies the superblock and inode structures to use the per-cpu >> list. The corresponding functions that reference those structures are >> modified. >> >> Waiman Long (2): >> lib/percpu-list: Per-cpu list with associated per-cpu locks >> vfs: Use per-cpu list for superblock's inode list > xfstests:xfs/013 deadlocks (running on 4GB ram disks on a 16p VM): > > [135478.644495] run fstests generic/013 at 2016-02-19 10:54:51 > [135479.058833] XFS (ram0): Unmounting Filesystem > [135479.149472] XFS (ram0): Mounting V5 Filesystem > [135479.154056] XFS (ram0): Ending clean mount > [135479.461571] XFS (ram0): Unmounting Filesystem > [135479.548060] XFS (ram0): Mounting V5 Filesystem > [135479.553103] XFS (ram0): Ending clean mount > [135507.633377] NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [fsstress:3390] > [135507.634572] Modules linked in: > [135507.635059] CPU: 4 PID: 3390 Comm: fsstress Not tainted 4.5.0-rc2-dgc+ #683 > [135507.636108] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Debian-1.8.2-1 04/01/2014 > [135507.637442] task: ffff88023a420000 ti: ffff88023758c000 task.ti: ffff88023758c000 > [135507.638566] RIP: 0010:[<ffffffff810f4982>] [<ffffffff810f4982>] do_raw_spin_lock+0x52/0x120 > [135507.639863] RSP: 0018:ffff88023758fe10 EFLAGS: 00000246 > [135507.640669] RAX: 0000000000000000 RBX: ffff880306e05ca8 RCX: 0000000000000024 > [135507.641751] RDX: 0000000000000001 RSI: 00010f15431142d8 RDI: ffff880306e05ca8 > [135507.642825] RBP: ffff88023758fe30 R08: 0000000000000004 R09: 0000000000000007 > [135507.643897] R10: 000000000002e400 R11: 0000000000000001 R12: ffffe8fefbd01800 > [135507.644966] R13: ffff880306e05db8 R14: ffff880306e05ca8 R15: ffff880306e05c20 > [135507.646051] FS: 00007f3f4244f700(0000) GS:ffff88013bc80000(0000) knlGS:0000000000000000 > [135507.647269] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [135507.648140] CR2: 00007f3f415cb008 CR3: 000000018c9ab000 CR4: 00000000000006e0 > [135507.649232] Stack: > [135507.649559] ffff880306e05c20 ffffe8fefbd01800 ffff880306e05db8 ffff880306e05ca8 > [135507.650740] ffff88023758fe40 ffffffff81e01d75 ffff88023758fee8 ffffffff8121727f > [135507.651924] ffff8802b2fb76b8 ffff8802b2fb7000 0000000a00000246 ffffe8fefbd01810 > [135507.653108] Call Trace: > [135507.653504] [<ffffffff81e01d75>] _raw_spin_lock+0x15/0x20 > [135507.654343] [<ffffffff8121727f>] sync_inodes_sb+0x1af/0x280 > [135507.655204] [<ffffffff8121d8f0>] ? SyS_tee+0x3d0/0x3d0 > [135507.656003] [<ffffffff8121d905>] sync_inodes_one_sb+0x15/0x20 > [135507.656891] [<ffffffff811efede>] iterate_supers+0xae/0x100 > [135507.657748] [<ffffffff8121dc35>] sys_sync+0x35/0x90 > [135507.658507] [<ffffffff81e0232e>] entry_SYSCALL_64_fastpath+0x12/0x71 > [135507.659483] Code: 00 00 48 39 43 10 0f 84 b7 00 00 00 65 8b 05 de 57 f1 7e 39 43 08 0f 84 bb 00 00 00 8b 03 85 c0 75 0d ba 01 00 00 00 f0 0f b1 13<85> c0 74 63 4c > [135507.733374] NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [fsstress:3380] > [135507.734665] Modules linked in: > [135507.735195] CPU: 10 PID: 3380 Comm: fsstress Tainted: G L 4.5.0-rc2-dgc+ #683 > [135507.736525] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Debian-1.8.2-1 04/01/2014 > [135507.737966] task: ffff8803814a23c0 ti: ffff880239264000 task.ti: ffff880239264000 > [135507.739176] RIP: 0010:[<ffffffff817da7b9>] [<ffffffff817da7b9>] delay_tsc+0x39/0x80 > [135507.740457] RSP: 0018:ffff880239267a00 EFLAGS: 00000206 > [135507.741338] RAX: 00010f23910bf4f8 RBX: ffffe8fefbd01810 RCX: 0000000000000024 > [135507.742523] RDX: 00010f2300000000 RSI: 00010f23910bf4d4 RDI: 0000000000000001 > [135507.743708] RBP: ffff880239267a00 R08: 000000000000000a R09: ffff880269028460 > [135507.744896] R10: 00000000c383dd1a R11: 00000000001f07e0 R12: 00000000334293fd > [135507.746087] R13: 0000000000000001 R14: 0000000083214e30 R15: ffff880265b51900 > [135507.747271] FS: 00007f3f4244f700(0000) GS:ffff88033bd00000(0000) knlGS:0000000000000000 > [135507.748604] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [135507.749568] CR2: 00007f3f4159d008 CR3: 0000000143196000 CR4: 00000000000006e0 > [135507.750753] Stack: > [135507.751111] ffff880239267a10 ffffffff817da6bf ffff880239267a40 ffffffff810f49bc > [135507.752400] 000060fbc0001800 ffffe8fefbd01810 ffff880306e07058 0000000000000000 > [135507.753697] ffff880239267a50 ffffffff81e01d75 ffff880239267a78 ffffffff817e529a > [135507.754991] Call Trace: > [135507.755424] [<ffffffff817da6bf>] __delay+0xf/0x20 > [135507.756234] [<ffffffff810f49bc>] do_raw_spin_lock+0x8c/0x120 > [135507.757204] [<ffffffff81e01d75>] _raw_spin_lock+0x15/0x20 > [135507.758130] [<ffffffff817e529a>] percpu_list_add+0x2a/0x70 > [135507.759064] [<ffffffff81206830>] inode_sb_list_add+0x20/0x30 > [135507.760025] [<ffffffff814f9744>] xfs_setup_inode+0x34/0x240 > [135507.760972] [<ffffffff814fb22b>] xfs_ialloc+0x36b/0x550 > [135507.761874] [<ffffffff814fb49d>] xfs_dir_ialloc+0x8d/0x260 > [135507.762805] [<ffffffff814fb92f>] xfs_create+0x25f/0x6a0 > [135507.763700] [<ffffffff814f881d>] xfs_generic_create+0xcd/0x2a0 > [135507.764686] [<ffffffff81e01dce>] ? _raw_spin_unlock+0xe/0x20 > [135507.765652] [<ffffffff814f8a26>] xfs_vn_create+0x16/0x20 > [135507.766556] [<ffffffff811f8ff2>] vfs_create+0xc2/0x120 > [135507.767430] [<ffffffff811fb6f9>] path_openat+0x1239/0x1370 > [135507.768364] [<ffffffff812b973c>] ? ext4_file_write_iter+0x21c/0x420 > [135507.769430] [<ffffffff810d2a89>] ? __might_sleep+0x49/0x80 > [135507.770362] [<ffffffff811fc8ee>] do_filp_open+0x7e/0xe0 > [135507.771256] [<ffffffff811e41f2>] ? kmem_cache_alloc+0x42/0x170 > [135507.772250] [<ffffffff81e01dce>] ? _raw_spin_unlock+0xe/0x20 > [135507.773225] [<ffffffff8120a2dc>] ? __alloc_fd+0xbc/0x170 > [135507.774135] [<ffffffff811eb2e6>] do_sys_open+0x116/0x1f0 > [135507.775043] [<ffffffff811eb41e>] SyS_creat+0x1e/0x20 > [135507.775895] [<ffffffff81e0232e>] entry_SYSCALL_64_fastpath+0x12/0x71 > > Cheers, > > Dave. Thanks for reporting this problem. PeterZ had actually noticed an issue in my patch that I used list_for_each_entry_safe() for all the modified s_inodes iteration functions which originally used list_for_each_entry(). That may not be safe in some cases. The hangup you saw may be caused by this problem. I am going to send out an updated patch to use the correct list_for_each_entry() macro for those iteration functions. Please try that out to see if you see the same hangup problem again. Cheers, Longman -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
