On 06/26/2013 11:20 PM, 符永涛 wrote: > Dear Brian and xfs experts, > > Recently we have encountered an issue related to xfs. The log reports xfs > soft lockup stuck which is similar to this bug: > http://bugs.centos.org/view.php?id=6287 > > I wan to ask a question. Is this issue already been fixed in kernel 358 or > not(BZ#921961, BZ#896224)? > Both of those bz's have metadata that shows the fixed-in release. The latter is a backport that notes the fix being pulled back into kernel-2.6.32-358.4.1.el6. Brian > > Thank you very much. > > logs: > Jun 26 17:57:06 cqdx kernel: BUG: soft lockup - CPU#0 stuck for 67s! > [xfslogd/0:2014] > Jun 26 17:57:06 cqdx kernel: Modules linked in: fuse ipv6 xfs exportfs > vhost_net macvtap macvlan tun kvm_intel kvm power_meter dcdbas microcode > sb_edac edac_core iTCO_wdt iTCO_vendor_support shpchp igb sg ixgbe dca mdio > ext4 mbcache jbd2 sd_mod crc_t10dif wmi megaraid_sas dm_mirror > dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan] > Jun 26 17:57:06 cqdx kernel: CPU 0 > Jun 26 17:57:06 cqdx kernel: Modules linked in: fuse ipv6 xfs exportfs > vhost_net macvtap macvlan tun kvm_intel kvm power_meter dcdbas microcode > sb_edac edac_core iTCO_wdt iTCO_vendor_support shpchp igb sg ixgbe dca mdio > ext4 mbcache jbd2 sd_mod crc_t10dif wmi megaraid_sas dm_mirror > dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan] > Jun 26 17:57:06 cqdx kernel: > Jun 26 17:57:06 cqdx kernel: Pid: 2014, comm: xfslogd/0 Not tainted > 2.6.32-279.19.1.01.qiyi.el6.x86_64 #1 Dell Inc. PowerEdge R720xd/0T0WRN > Jun 26 17:57:06 cqdx kernel: RIP: 0010:[<ffffffff814ec55c>] > [<ffffffff814ec55c>] _spin_lock+0x1c/0x30 > Jun 26 17:57:06 cqdx kernel: RSP: 0000:ffff88101465bd50 EFLAGS: 00000293 > Jun 26 17:57:06 cqdx kernel: RAX: 0000000000005468 RBX: ffff88101465bd50 > RCX: ffffe8efaf600e88 > Jun 26 17:57:06 cqdx kernel: RDX: 0000000000005466 RSI: ffff882011755898 > RDI: ffff88201955b100 > Jun 26 17:57:06 cqdx kernel: RBP: ffffffff8100bb8e R08: 0000000000000000 > R09: 00000000ffffffff > Jun 26 17:57:06 cqdx kernel: R10: 0000000000000000 R11: 0000000000000000 > R12: ffff88101465bd70 > Jun 26 17:57:06 cqdx kernel: R13: ffffffff8100b9ce R14: 0000000000000400 > R15: 0000000000000094 > Jun 26 17:57:06 cqdx kernel: FS: 0000000000000000(0000) > GS:ffff880028200000(0000) knlGS:0000000000000000 > Jun 26 17:57:06 cqdx kernel: CS: 0010 DS: 0018 ES: 0018 CR0: > 000000008005003b > Jun 26 17:57:06 cqdx kernel: CR2: 00007f9e001dc0a0 CR3: 000000201311e000 > CR4: 00000000000406f0 > Jun 26 17:57:06 cqdx kernel: DR0: 0000000000000000 DR1: 0000000000000000 > DR2: 0000000000000000 > Jun 26 17:57:06 cqdx kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 > DR7: 0000000000000400 > Jun 26 17:57:06 cqdx kernel: Process xfslogd/0 (pid: 2014, threadinfo > ffff88101465a000, task ffff881015b75500) > Jun 26 17:57:06 cqdx kernel: Stack: > Jun 26 17:57:06 cqdx kernel: ffff88101465bd80 ffffffffa02a2b3b > 0000000000000001 ffff8820137c7300 > Jun 26 17:57:06 cqdx kernel: <d> ffff882016daee40 ffff882017e03800 > ffff88101465bda0 ffffffffa02a2adc > Jun 26 17:57:06 cqdx kernel: <d> ffff882016d330a8 ffff882016daee40 > ffff88101465be10 ffffffffa02a2d51 > Jun 26 17:57:06 cqdx kernel: Call Trace: > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02a2b3b>] ? > xfs_buf_iodone+0x2b/0x60 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02a2adc>] ? > xfs_buf_do_callbacks+0x3c/0x50 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02a2d51>] ? > xfs_buf_iodone_callbacks+0x51/0x270 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02db190>] ? > xfs_buf_iodone_work+0x0/0x50 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02db1b6>] ? > xfs_buf_iodone_work+0x26/0x50 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffff8108b120>] ? > worker_thread+0x170/0x2a0 > Jun 26 17:57:06 cqdx kernel: [<ffffffff81090990>] ? > autoremove_wake_function+0x0/0x40 > Jun 26 17:57:06 cqdx kernel: [<ffffffff8108afb0>] ? worker_thread+0x0/0x2a0 > Jun 26 17:57:06 cqdx kernel: [<ffffffff81090626>] ? kthread+0x96/0xa0 > Jun 26 17:57:06 cqdx kernel: [<ffffffff8100c0ca>] ? child_rip+0xa/0x20 > Jun 26 17:57:06 cqdx kernel: [<ffffffff81090590>] ? kthread+0x0/0xa0 > Jun 26 17:57:06 cqdx kernel: [<ffffffff8100c0c0>] ? child_rip+0x0/0x20 > Jun 26 17:57:06 cqdx kernel: Code: 81 2f 00 00 00 01 74 05 e8 72 8c d8 ff > c9 c3 55 48 89 e5 0f 1f 44 00 00 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 > 10 39 c2 74 0e <f3> 90 0f b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 > 00 55 > Jun 26 17:57:06 cqdx kernel: Call Trace: > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02a2b3b>] ? > xfs_buf_iodone+0x2b/0x60 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02a2adc>] ? > xfs_buf_do_callbacks+0x3c/0x50 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02a2d51>] ? > xfs_buf_iodone_callbacks+0x51/0x270 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02db190>] ? > xfs_buf_iodone_work+0x0/0x50 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02db1b6>] ? > xfs_buf_iodone_work+0x26/0x50 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffff8108b120>] ? > worker_thread+0x170/0x2a0 > Jun 26 17:57:06 cqdx kernel: [<ffffffff81090990>] ? > autoremove_wake_function+0x0/0x40 > Jun 26 17:57:06 cqdx kernel: [<ffffffff8108afb0>] ? worker_thread+0x0/0x2a0 > Jun 26 17:57:06 cqdx kernel: [<ffffffff81090626>] ? kthread+0x96/0xa0 > Jun 26 17:57:06 cqdx kernel: [<ffffffff8100c0ca>] ? child_rip+0xa/0x20 > Jun 26 17:57:06 cqdx kernel: [<ffffffff81090590>] ? kthread+0x0/0xa0 > Jun 26 17:57:06 cqdx kernel: [<ffffffff8100c0c0>] ? child_rip+0x0/0x20 > Jun 26 17:57:06 cqdx kernel: BUG: soft lockup - CPU#15 stuck for 67s! > [glusterfsd:3245] > Jun 26 17:57:06 cqdx kernel: Modules linked in: fuse ipv6 xfs exportfs > vhost_net macvtap macvlan tun kvm_intel kvm power_meter dcdbas microcode > sb_edac edac_core iTCO_wdt iTCO_vendor_support shpchp igb sg ixgbe dca mdio > ext4 mbcache jbd2 sd_mod crc_t10dif wmi megaraid_sas dm_mirror > dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan] > Jun 26 17:57:06 cqdx kernel: CPU 15 > Jun 26 17:57:06 cqdx kernel: Modules linked in: fuse ipv6 xfs exportfs > vhost_net macvtap macvlan tun kvm_intel kvm power_meter dcdbas microcode > sb_edac edac_core iTCO_wdt iTCO_vendor_support shpchp igb sg ixgbe dca mdio > ext4 mbcache jbd2 sd_mod crc_t10dif wmi megaraid_sas dm_mirror > dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan] > Jun 26 17:57:06 cqdx kernel: > Jun 26 17:57:06 cqdx kernel: Pid: 3245, comm: glusterfsd Not tainted > 2.6.32-279.19.1.01.qiyi.el6.x86_64 #1 Dell Inc. PowerEdge R720xd/0T0WRN > Jun 26 17:57:06 cqdx kernel: RIP: 0010:[<ffffffff814ec55e>] > [<ffffffff814ec55e>] _spin_lock+0x1e/0x30 > Jun 26 17:57:06 cqdx kernel: RSP: 0018:ffff880dceb5d528 EFLAGS: 00000297 > Jun 26 17:57:06 cqdx kernel: RAX: 00000000000087eb RBX: ffff880dceb5d528 > RCX: ffff8820137923e0 > Jun 26 17:57:06 cqdx kernel: RDX: 00000000000087ea RSI: 0000000000006e5c > RDI: ffff882016d330a8 > Jun 26 17:57:06 cqdx kernel: RBP: ffffffff8100bb8e R08: ffff880dceb5d674 > R09: ffff880dceb5d670 > Jun 26 17:57:06 cqdx kernel: R10: 0000000000000000 R11: 0000000000000000 > R12: 0000000000000000 > Jun 26 17:57:06 cqdx kernel: R13: 0000000000000cb5 R14: ffff8810788166e8 > R15: ffff882016ea9400 > Jun 26 17:57:06 cqdx kernel: FS: 00007f6cf3fff700(0000) > GS:ffff8810788e0000(0000) knlGS:0000000000000000 > Jun 26 17:57:06 cqdx kernel: CS: 0010 DS: 0000 ES: 0000 CR0: > 0000000080050033 > Jun 26 17:57:06 cqdx kernel: CR2: 0000000000481046 CR3: 000000201311e000 > CR4: 00000000000406e0 > Jun 26 17:57:06 cqdx kernel: DR0: 0000000000000000 DR1: 0000000000000000 > DR2: 0000000000000000 > Jun 26 17:57:06 cqdx kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 > DR7: 0000000000000400 > Jun 26 17:57:06 cqdx kernel: Process glusterfsd (pid: 3245, threadinfo > ffff880dceb5c000, task ffff8810191ecaa0) > Jun 26 17:57:06 cqdx kernel: Stack: > Jun 26 17:57:06 cqdx kernel: ffff880dceb5d5d8 ffffffffa02c27a8 > ffff881078816680 ffff880dceb5d670 > Jun 26 17:57:06 cqdx kernel: <d> ffff880dceb5d674 ffff880dceb5d668 > 00006e5c0000000f 0000000000000001 > Jun 26 17:57:06 cqdx kernel: <d> 00000000000172a0 ffff882016d33080 > ffff880dceb5d588 0000000000000082 > Jun 26 17:57:06 cqdx kernel: Call Trace: > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02c27a8>] ? > xlog_state_get_iclog_space+0x68/0x2e0 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffff8105f8ac>] ? > try_to_wake_up+0x24c/0x3e0 > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02c2bf1>] ? xlog_write+0x181/0x6d0 > [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffff8126f9f5>] ? rwsem_wake+0x75/0x170 > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02c3c41>] ? > xlog_cil_push+0x261/0x3d0 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02c43a5>] ? > xlog_cil_force_lsn+0x115/0x120 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa029a30d>] ? > xfs_bmap_add_extent+0xad/0x3c0 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02c2308>] ? > _xfs_log_force+0x68/0x240 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02c2718>] ? xfs_log_force+0x18/0x40 > [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02d99d0>] ? > xfs_buf_cond_lock+0x90/0xc0 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02dadb3>] ? > _xfs_buf_find+0xe3/0x280 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02dafbb>] ? xfs_buf_get+0x6b/0x1a0 > [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02d16d8>] ? > xfs_trans_get_buf+0xe8/0x180 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02a4a67>] ? > xfs_da_do_buf+0x4c7/0x770 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02a5fda>] ? > xfs_da_get_buf+0x2a/0x30 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02aa49d>] ? > xfs_dir2_data_init+0x4d/0xf0 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02aa49d>] ? > xfs_dir2_data_init+0x4d/0xf0 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02a8c23>] ? > xfs_dir2_sf_to_block+0xe3/0x5e0 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02b5ce6>] ? xfs_iunlock+0x86/0x100 > [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffff8118ecaa>] ? dput+0x9a/0x150 > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02b0c18>] ? > xfs_dir2_sf_addname+0x458/0x5b0 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02d6687>] ? > kmem_zone_alloc+0x77/0xf0 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02a81bc>] ? > xfs_dir_createname+0x14c/0x1b0 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02d3390>] ? xfs_link+0x1f0/0x300 > [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffff81195560>] ? > mntput_no_expire+0x30/0x110 > Jun 26 17:57:06 cqdx kernel: [<ffffffff8118e7a7>] ? __d_lookup+0xa7/0x150 > Jun 26 17:57:06 cqdx kernel: [<ffffffff8107721a>] ? capable+0x2a/0x60 > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02e1120>] ? xfs_vn_link+0x40/0x70 > [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffff8118379c>] ? vfs_link+0x11c/0x190 > Jun 26 17:57:06 cqdx kernel: [<ffffffff81182aaa>] ? lookup_hash+0x3a/0x50 > Jun 26 17:57:06 cqdx kernel: [<ffffffff81186b71>] ? sys_linkat+0x141/0x160 > Jun 26 17:57:06 cqdx kernel: [<ffffffff8117b83b>] ? vfs_stat+0x1b/0x20 > Jun 26 17:57:06 cqdx kernel: [<ffffffff8117b864>] ? sys_newstat+0x24/0x50 > Jun 26 17:57:06 cqdx kernel: [<ffffffff81186bae>] ? sys_link+0x1e/0x20 > Jun 26 17:57:06 cqdx kernel: [<ffffffff8100b072>] ? > system_call_fastpath+0x16/0x1b > Jun 26 17:57:06 cqdx kernel: Code: 00 00 00 01 74 05 e8 72 8c d8 ff c9 c3 > 55 48 89 e5 0f 1f 44 00 00 b8 00 00 01 00 f0 0f c1 07 0f b7 d0 c1 e8 10 39 > c2 74 0e f3 90 <0f> b7 17 eb f5 83 3f 00 75 f4 eb df c9 c3 0f 1f 40 00 55 > 48 89 > Jun 26 17:57:06 cqdx kernel: Call Trace: > Jun 26 17:57:06 cqdx kernel: [<ffffffff8106210b>] ? > enqueue_task_fair+0xfb/0x100 > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02c27a8>] ? > xlog_state_get_iclog_space+0x68/0x2e0 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffff8105f8ac>] ? > try_to_wake_up+0x24c/0x3e0 > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02c2bf1>] ? xlog_write+0x181/0x6d0 > [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffff8126f9f5>] ? rwsem_wake+0x75/0x170 > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02c3c41>] ? > xlog_cil_push+0x261/0x3d0 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa02c43a5>] ? > xlog_cil_force_lsn+0x115/0x120 [xfs] > Jun 26 17:57:06 cqdx kernel: [<ffffffffa029a30d>] ? > xfs_bmap_add_extent+0xad/0x3c0 [xfs] > > _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
