Hello, after upgrading from 2.6.39.1 in December, I've been having freezes with 3.1.x, 3.2.2. In an attempt to find the cause, when upgrading to 3.3.1 in the beginning of April I enabled all the debug settings I could find in the kernel, but it never froze and I assumed that the bug had been fixed. 4-5 days ago I upgraded to 3.4.3 without debug flags (they made everything horribly slow), and today it froze again! I can ssh in, but don't get further than "Last login: Tue Jun 26 11:26:10 CEST 2012 from 10.0.0.2 on pts/4". I assume it tries to write new info to /var/log/ and hangs. I got around 10 disks connected to the Intel motherboard controller (ahci) and two sata_sil24 PCI-e cards. They're not partitioned very wisely: All the disks have a cryptsetup partition. Mdadm runs on top of the encrypted volumes. Lvm2 on the big /dev/md0 device. With xfs filesystems inside. And this is also served over nfs. That is: cryptsetup -> mdadm -> lvm2 -> xfs (-> nfs). Here is Show Blocked State from netconsole. To me it looks like the problem might be in get_active_stripe, but I don't understand much of threads/synchronization. Jun 26 22:27:29 [368353.502333] SysRq : Show Blocked State Jun 26 22:27:29 [368353.502414] task PC stack pid father Jun 26 22:27:29 [368353.502478] flush-254:12 D ffff88010db48cc0 0 5522 2 0x00000000 Jun 26 22:27:29 [368353.502600] ffff88010cfc9640 0000000000000046 ffff88010cfc95d0 ffff88010c3ccc80 Jun 26 22:27:29 [368353.502764] ffff88010f0029c0 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.502924] ffff88010db48cc0 00000000000107c0 ffff88010cfc9fd8 ffff88010cfc9fd8 Jun 26 22:27:29 [368353.503085] Call Trace: Jun 26 22:27:29 [368353.503127] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.503176] [<ffffffffa00b38ff>] get_active_stripe+0x256/0x564 [raid456] Jun 26 22:27:29 [368353.503220] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:29 [368353.503392] [<ffffffffa00b5976>] make_request+0x1dd/0x3bc [raid456] Jun 26 22:27:29 [368353.503564] [<ffffffff8103ffc0>] ? wake_up_bit+0x25/0x25 Jun 26 22:27:29 [368353.503738] [<ffffffffa0069ef1>] md_make_request+0xd0/0x1c2 [md_mod] Jun 26 22:27:29 [368353.503970] [<ffffffff811c6b93>] ? dm_request+0x1b4/0x1c3 Jun 26 22:27:29 [368353.504140] [<ffffffff811c9c8c>] ? linear_merge+0x4d/0x58 Jun 26 22:27:29 [368353.504310] [<ffffffff81119ad1>] generic_make_request+0x97/0xd9 Jun 26 22:27:29 [368353.504480] [<ffffffff81119bee>] submit_bio+0xdb/0xfa Jun 26 22:27:29 [368353.504659] [<ffffffffa01023e5>] xfs_submit_ioend_bio.clone.19+0x2e/0x30 [xfs] Jun 26 22:27:29 [368353.504967] [<ffffffffa01024b5>] xfs_submit_ioend+0xce/0xee [xfs] Jun 26 22:27:29 [368353.505144] [<ffffffffa0102859>] xfs_vm_writepage+0x384/0x412 [xfs] Jun 26 22:27:29 [368353.505316] [<ffffffff8108a23e>] __writepage+0x12/0x2b Jun 26 22:27:29 [368353.505485] [<ffffffff8108aa5c>] write_cache_pages+0x225/0x377 Jun 26 22:27:29 [368353.505656] [<ffffffff8108a22c>] ? set_page_dirty+0x5b/0x5b Jun 26 22:27:29 [368353.505826] [<ffffffff8108abee>] generic_writepages+0x40/0x57 Jun 26 22:27:29 [368353.506001] [<ffffffffa0101c06>] xfs_vm_writepages+0x47/0x4e [xfs] Jun 26 22:27:29 [368353.506172] [<ffffffff8108bbec>] do_writepages+0x1c/0x25 Jun 26 22:27:29 [368353.506342] [<ffffffff810d1c17>] writeback_single_inode+0xcf/0x232 Jun 26 22:27:29 [368353.506513] [<ffffffff810d2015>] writeback_sb_inodes+0x163/0x1fe Jun 26 22:27:29 [368353.506684] [<ffffffff810d240e>] __writeback_inodes_wb+0x6d/0xab Jun 26 22:27:29 [368353.506854] [<ffffffff810d26dd>] wb_writeback+0xfc/0x182 Jun 26 22:27:29 [368353.507024] [<ffffffff810c91b9>] ? get_nr_inodes_unused+0x43/0x5a Jun 26 22:27:29 [368353.507195] [<ffffffff8125f1d9>] ? _raw_spin_lock_irqsave+0xd/0x27 Jun 26 22:27:29 [368353.507365] [<ffffffff810d2dbd>] wb_do_writeback+0x111/0x16d Jun 26 22:27:29 [368353.507537] [<ffffffff81033322>] ? run_timer_softirq+0x1ef/0x1ef Jun 26 22:27:29 [368353.507707] [<ffffffff810d2e9e>] bdi_writeback_thread+0x85/0x14a Jun 26 22:27:29 [368353.507877] [<ffffffff810d2e19>] ? wb_do_writeback+0x16d/0x16d Jun 26 22:27:29 [368353.508047] [<ffffffff810d2e19>] ? wb_do_writeback+0x16d/0x16d Jun 26 22:27:29 [368353.508217] [<ffffffff8103fba9>] kthread+0x84/0x8c Jun 26 22:27:29 [368353.508386] [<ffffffff81260b54>] kernel_thread_helper+0x4/0x10 Jun 26 22:27:29 [368353.508556] [<ffffffff8103fb25>] ? kthread_freezable_should_stop+0x41/0x41 Jun 26 22:27:29 [368353.508728] [<ffffffff81260b50>] ? gs_change+0xb/0xb Jun 26 22:27:29 [368353.508899] dhcpcd D ffff88010b2d2ca0 0 6706 1 0x00000000 Jun 26 22:27:29 [368353.509131] ffff88010c8c7828 0000000000000082 ffff88010c8c77b8 ffff88010dd1d940 Jun 26 22:27:29 [368353.509636] 0000000000000005 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.510056] ffff88010b2d2ca0 00000000000107c0 ffff88010c8c7fd8 ffff88010c8c7fd8 Jun 26 22:27:29 [368353.510475] Call Trace: Jun 26 22:27:29 [368353.510640] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.510811] [<ffffffffa00b38ff>] get_active_stripe+0x256/0x564 [raid456] Jun 26 22:27:29 [368353.510982] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:29 [368353.511153] [<ffffffffa00b5976>] make_request+0x1dd/0x3bc [raid456] Jun 26 22:27:29 [368353.511323] [<ffffffff8103ffc0>] ? wake_up_bit+0x25/0x25 Jun 26 22:27:29 [368353.511496] [<ffffffffa0069ef1>] md_make_request+0xd0/0x1c2 [md_mod] Jun 26 22:27:29 [368353.511666] [<ffffffff811c6b93>] ? dm_request+0x1b4/0x1c3 Jun 26 22:27:29 [368353.511835] [<ffffffff811c9c8c>] ? linear_merge+0x4d/0x58 Jun 26 22:27:29 [368353.512005] [<ffffffff81119ad1>] generic_make_request+0x97/0xd9 Jun 26 22:27:29 [368353.512176] [<ffffffff81119bee>] submit_bio+0xdb/0xfa Jun 26 22:27:29 [368353.512325] [<ffffffffa01023e5>] xfs_submit_ioend_bio.clone.19+0x2e/0x30 [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffffa01024b5>] xfs_submit_ioend+0xce/0xee [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffffa0102859>] xfs_vm_writepage+0x384/0x412 [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffff8108a23e>] __writepage+0x12/0x2b Jun 26 22:27:29 [368353.512325] [<ffffffff8108aa5c>] write_cache_pages+0x225/0x377 Jun 26 22:27:29 [368353.512325] [<ffffffff8108a22c>] ? set_page_dirty+0x5b/0x5b Jun 26 22:27:29 [368353.512325] [<ffffffffa0105bb2>] ? xfs_rw_iunlock+0x24/0x29 [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffffa01067c5>] ? xfs_file_buffered_aio_write+0xfd/0x110 [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffff8108abee>] generic_writepages+0x40/0x57 Jun 26 22:27:29 [368353.512325] [<ffffffffa0101c06>] xfs_vm_writepages+0x47/0x4e [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffff8108bbec>] do_writepages+0x1c/0x25 Jun 26 22:27:29 [368353.512325] [<ffffffff81084a71>] __filemap_fdatawrite_range+0x4e/0x50 Jun 26 22:27:29 [368353.512325] [<ffffffff810855e3>] filemap_fdatawrite_range+0xe/0x10 Jun 26 22:27:29 [368353.512325] [<ffffffffa010822a>] xfs_flush_pages+0x5c/0x8e [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffffa0112ebd>] xfs_release+0xe8/0x1ab [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffffa0105eb4>] xfs_file_release+0x10/0x14 [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffff810b729c>] fput+0xf9/0x1bd Jun 26 22:27:29 [368353.512325] [<ffffffff810b449a>] filp_close+0x6b/0x76 Jun 26 22:27:29 [368353.512325] [<ffffffff810b4523>] sys_close+0x7e/0xbb Jun 26 22:27:29 [368353.512325] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.512325] nfsd D ffffffffffffffff 0 7164 2 0x00000000 Jun 26 22:27:29 [368353.512325] ffff880107f99b18 0000000000000046 ffff88010a330d80 ffff88010dd1bfc0 Jun 26 22:27:29 [368353.512325] ffffffff810c7402 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.512325] ffff88010c8ea640 00000000000107c0 ffff880107f99fd8 ffff880107f99fd8 Jun 26 22:27:29 [368353.512325] Call Trace: Jun 26 22:27:29 [368353.512325] [<ffffffff810c7402>] ? d_obtain_alias+0x142/0x14e Jun 26 22:27:29 [368353.512325] [<ffffffffa0105975>] ? xfs_dir_fsync+0x77/0x77 [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.512325] [<ffffffff8125efa0>] rwsem_down_failed_common+0xd7/0x109 Jun 26 22:27:29 [368353.512325] [<ffffffff8125eff9>] rwsem_down_read_failed+0x12/0x14 Jun 26 22:27:29 [368353.512325] [<ffffffff81136a84>] call_rwsem_down_read_failed+0x14/0x30 Jun 26 22:27:29 [368353.512325] [<ffffffff810b6eb8>] ? get_empty_filp+0x66/0x113 Jun 26 22:27:29 [368353.512325] [<ffffffff8125de47>] ? down_read+0x12/0x14 Jun 26 22:27:29 [368353.512325] [<ffffffffa0108483>] xfs_ilock+0x59/0x5e [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffffa01084b8>] xfs_ilock_map_shared+0x30/0x36 [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffffa010599e>] xfs_dir_open+0x29/0x58 [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffff810b4899>] __dentry_open.clone.14+0x149/0x263 Jun 26 22:27:29 [368353.512325] [<ffffffff810b4a04>] dentry_open+0x51/0x5a Jun 26 22:27:29 [368353.512325] [<ffffffffa0315c76>] nfsd_open+0x11f/0x158 [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffffa0316797>] nfsd_readdir+0x43/0x1cf [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffffa031d750>] ? nfs3svc_encode_entry+0xe/0xe [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffffa027b4b7>] ? cache_check+0x1f4/0x216 [sunrpc] Jun 26 22:27:29 [368353.512325] [<ffffffffa031b59c>] nfsd3_proc_readdirplus+0x8b/0x16f [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffffa031279e>] nfsd_dispatch+0xa5/0x15c [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffffa0272b39>] svc_process+0x3a5/0x567 [sunrpc] Jun 26 22:27:29 [368353.512325] [<ffffffffa031205f>] ? nfsd_get_default_max_blksize+0x3a/0x3a [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffffa0312149>] nfsd+0xea/0x12e [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffffa031205f>] ? nfsd_get_default_max_blksize+0x3a/0x3a [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffff8103fba9>] kthread+0x84/0x8c Jun 26 22:27:29 [368353.512325] [<ffffffff81260b54>] kernel_thread_helper+0x4/0x10 Jun 26 22:27:29 [368353.512325] [<ffffffff8103fb25>] ? kthread_freezable_should_stop+0x41/0x41 Jun 26 22:27:29 [368353.512325] [<ffffffff81260b50>] ? gs_change+0xb/0xb Jun 26 22:27:29 [368353.512325] nfsd D ffffffffffffffff 0 7166 2 0x00000000 Jun 26 22:27:29 [368353.512325] ffff8801028a5b18 0000000000000046 ffff88010a330d80 ffff88010dd1bfc0 Jun 26 22:27:29 [368353.512325] ffffffff810c7402 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.512325] ffff88010c8e8660 00000000000107c0 ffff8801028a5fd8 ffff8801028a5fd8 Jun 26 22:27:29 [368353.512325] Call Trace: Jun 26 22:27:29 [368353.512325] [<ffffffff810c7402>] ? d_obtain_alias+0x142/0x14e Jun 26 22:27:29 [368353.512325] [<ffffffffa0105975>] ? xfs_dir_fsync+0x77/0x77 [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.512325] [<ffffffff8125efa0>] rwsem_down_failed_common+0xd7/0x109 Jun 26 22:27:29 [368353.512325] [<ffffffff8125eff9>] rwsem_down_read_failed+0x12/0x14 Jun 26 22:27:29 [368353.512325] [<ffffffff81136a84>] call_rwsem_down_read_failed+0x14/0x30 Jun 26 22:27:29 [368353.512325] [<ffffffff810b6eb8>] ? get_empty_filp+0x66/0x113 Jun 26 22:27:29 [368353.512325] [<ffffffff8125de47>] ? down_read+0x12/0x14 Jun 26 22:27:29 [368353.512325] [<ffffffffa0108483>] xfs_ilock+0x59/0x5e [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffffa01084b8>] xfs_ilock_map_shared+0x30/0x36 [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffffa010599e>] xfs_dir_open+0x29/0x58 [xfs] Jun 26 22:27:29 [368353.512325] [<ffffffff810b4899>] __dentry_open.clone.14+0x149/0x263 Jun 26 22:27:29 [368353.512325] [<ffffffff810b4a04>] dentry_open+0x51/0x5a Jun 26 22:27:29 [368353.512325] [<ffffffffa0315c76>] nfsd_open+0x11f/0x158 [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffffa0316797>] nfsd_readdir+0x43/0x1cf [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffffa031d750>] ? nfs3svc_encode_entry+0xe/0xe [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffffa027b4b7>] ? cache_check+0x1f4/0x216 [sunrpc] Jun 26 22:27:29 [368353.512325] [<ffffffffa031b59c>] nfsd3_proc_readdirplus+0x8b/0x16f [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffffa031279e>] nfsd_dispatch+0xa5/0x15c [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffffa0272b39>] svc_process+0x3a5/0x567 [sunrpc] Jun 26 22:27:29 [368353.512325] [<ffffffffa031205f>] ? nfsd_get_default_max_blksize+0x3a/0x3a [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffffa0312149>] nfsd+0xea/0x12e [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffffa031205f>] ? nfsd_get_default_max_blksize+0x3a/0x3a [nfsd] Jun 26 22:27:29 [368353.512325] [<ffffffff8103fba9>] kthread+0x84/0x8c Jun 26 22:27:29 [368353.512325] [<ffffffff81260b54>] kernel_thread_helper+0x4/0x10 Jun 26 22:27:29 [368353.512325] [<ffffffff8103fb25>] ? kthread_freezable_should_stop+0x41/0x41 Jun 26 22:27:29 [368353.512325] [<ffffffff81260b50>] ? gs_change+0xb/0xb Jun 26 22:27:29 [368353.512325] agetty D ffff880101e3bb18 0 7323 1 0x00000000 Jun 26 22:27:29 [368353.512325] ffff880101e3ba68 0000000000000082 ffff880101e3b9e8 ffff88010dd1d940 Jun 26 22:27:29 [368353.530119] ffff880101e3ba18 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010c8bb960 00000000000107c0 ffff880101e3bfd8 ffff880101e3bfd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8108403a>] ? sleep_on_page+0xd/0xd Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e81e>] io_schedule+0x5e/0x79 Jun 26 22:27:29 [368353.530119] [<ffffffff81084043>] sleep_on_page_killable+0x9/0x23 Jun 26 22:27:29 [368353.530119] [<ffffffff8125d7cb>] __wait_on_bit_lock+0x41/0x8a Jun 26 22:27:29 [368353.530119] [<ffffffff81083f77>] __lock_page_killable+0x61/0x68 Jun 26 22:27:29 [368353.530119] [<ffffffff8103fff4>] ? autoremove_wake_function+0x34/0x34 Jun 26 22:27:29 [368353.530119] [<ffffffff8113144a>] ? radix_tree_lookup_slot+0xe/0x10 Jun 26 22:27:29 [368353.530119] [<ffffffff81083f9f>] lock_page_killable+0x21/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffff810853a3>] generic_file_aio_read+0x454/0x61e Jun 26 22:27:29 [368353.530119] [<ffffffff8104ac69>] ? update_curr+0x42/0x79 Jun 26 22:27:29 [368353.530119] [<ffffffffa01062a1>] xfs_file_aio_read+0x1a4/0x1fa [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810b592c>] do_sync_read+0xc9/0x106 Jun 26 22:27:29 [368353.530119] [<ffffffffa0110fdf>] ? xfs_xattr_get+0x37/0x40 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810cf77c>] ? generic_getxattr+0x4e/0x5b Jun 26 22:27:29 [368353.530119] [<ffffffff811166b4>] ? get_vfs_caps_from_disk+0x54/0xcd Jun 26 22:27:29 [368353.530119] [<ffffffff8125f01c>] ? _raw_spin_lock+0x9/0xb Jun 26 22:27:29 [368353.530119] [<ffffffff810b6032>] vfs_read+0xa4/0xf7 Jun 26 22:27:29 [368353.530119] [<ffffffff810ba035>] kernel_read+0x3c/0x4c Jun 26 22:27:29 [368353.530119] [<ffffffff810ba126>] prepare_binprm+0xe1/0xe5 Jun 26 22:27:29 [368353.530119] [<ffffffff810bb998>] do_execve_common.clone.35+0x223/0x392 Jun 26 22:27:29 [368353.530119] [<ffffffff810bbb1d>] do_execve+0x16/0x18 Jun 26 22:27:29 [368353.530119] [<ffffffff81009c54>] sys_execve+0x3e/0x55 Jun 26 22:27:29 [368353.530119] [<ffffffff8125fe2c>] stub_execve+0x6c/0xc0 Jun 26 22:27:29 [368353.530119] agetty D ffff88010c8b8000 0 7324 1 0x00000000 Jun 26 22:27:29 [368353.530119] ffff880101e5b588 0000000000000086 ffff880101e5b518 ffff88010dd1f2c0 Jun 26 22:27:29 [368353.530119] 0000000000000400 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010c8b8000 00000000000107c0 ffff880101e5bfd8 ffff880101e5bfd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffffa00b38ff>] get_active_stripe+0x256/0x564 [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:29 [368353.530119] [<ffffffffa00b5976>] make_request+0x1dd/0x3bc [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff8103ffc0>] ? wake_up_bit+0x25/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffffa0069ef1>] md_make_request+0xd0/0x1c2 [md_mod] Jun 26 22:27:29 [368353.530119] [<ffffffff811c6b93>] ? dm_request+0x1b4/0x1c3 Jun 26 22:27:29 [368353.530119] [<ffffffff81119ad1>] generic_make_request+0x97/0xd9 Jun 26 22:27:29 [368353.530119] [<ffffffff81119bee>] submit_bio+0xdb/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffff810defa8>] do_mpage_readpage+0x443/0x4f7 Jun 26 22:27:29 [368353.530119] [<ffffffff8108cb59>] ? __lru_cache_add+0x2f/0x56 Jun 26 22:27:29 [368353.530119] [<ffffffff810df1aa>] mpage_readpages+0xe9/0x134 Jun 26 22:27:29 [368353.530119] [<ffffffffa0101a00>] ? xfs_get_blocks_direct+0x11/0x11 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa01037a4>] ? xfs_buf_rele+0x32/0x8c [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0101a00>] ? xfs_get_blocks_direct+0x11/0x11 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0101bbd>] xfs_vm_readpages+0x18/0x1a [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108be39>] __do_page_cache_readahead+0x135/0x1d2 Jun 26 22:27:29 [368353.530119] [<ffffffff8108c14c>] ra_submit+0x1c/0x20 Jun 26 22:27:29 [368353.530119] [<ffffffff8108c307>] ondemand_readahead+0x1b7/0x1c6 Jun 26 22:27:29 [368353.530119] [<ffffffff810c5f00>] ? dentry_update_name_case+0x1c/0x56 Jun 26 22:27:29 [368353.530119] [<ffffffff8108c3cb>] page_cache_sync_readahead+0x38/0x3d Jun 26 22:27:29 [368353.530119] [<ffffffff810851ec>] generic_file_aio_read+0x29d/0x61e Jun 26 22:27:29 [368353.530119] [<ffffffff8104ac69>] ? update_curr+0x42/0x79 Jun 26 22:27:29 [368353.530119] [<ffffffffa01062a1>] xfs_file_aio_read+0x1a4/0x1fa [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810b592c>] do_sync_read+0xc9/0x106 Jun 26 22:27:29 [368353.530119] [<ffffffffa0110fdf>] ? xfs_xattr_get+0x37/0x40 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810cf77c>] ? generic_getxattr+0x4e/0x5b Jun 26 22:27:29 [368353.530119] [<ffffffff811166b4>] ? get_vfs_caps_from_disk+0x54/0xcd Jun 26 22:27:29 [368353.530119] [<ffffffff8125f01c>] ? _raw_spin_lock+0x9/0xb Jun 26 22:27:29 [368353.530119] [<ffffffff810b6032>] vfs_read+0xa4/0xf7 Jun 26 22:27:29 [368353.530119] [<ffffffff810ba035>] kernel_read+0x3c/0x4c Jun 26 22:27:29 [368353.530119] [<ffffffff810ba126>] prepare_binprm+0xe1/0xe5 Jun 26 22:27:29 [368353.530119] [<ffffffff810bb998>] do_execve_common.clone.35+0x223/0x392 Jun 26 22:27:29 [368353.530119] [<ffffffff810bbb1d>] do_execve+0x16/0x18 Jun 26 22:27:29 [368353.530119] [<ffffffff81009c54>] sys_execve+0x3e/0x55 Jun 26 22:27:29 [368353.530119] [<ffffffff8125fe2c>] stub_execve+0x6c/0xc0 Jun 26 22:27:29 [368353.530119] irssi D ffffffff8108402d 0 7343 1 0x00000000 Jun 26 22:27:29 [368353.530119] ffff880100487a38 0000000000000082 ffff8801004879b8 ffff88010c3c8cc0 Jun 26 22:27:29 [368353.530119] ffff8801004879e8 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010c8bd940 00000000000107c0 ffff880100487fd8 ffff880100487fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8108402d>] ? lock_page+0x22/0x22 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e81e>] io_schedule+0x5e/0x79 Jun 26 22:27:29 [368353.530119] [<ffffffff81084036>] sleep_on_page+0x9/0xd Jun 26 22:27:29 [368353.530119] [<ffffffff8125d8cb>] __wait_on_bit+0x43/0x76 Jun 26 22:27:29 [368353.530119] [<ffffffffa0140b02>] ? xfs_trans_free_item_desc+0x2e/0x30 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810841e3>] wait_on_page_bit+0x6d/0x74 Jun 26 22:27:29 [368353.530119] [<ffffffff8103fff4>] ? autoremove_wake_function+0x34/0x34 Jun 26 22:27:29 [368353.530119] [<ffffffff81084109>] ? find_lock_page+0x32/0x56 Jun 26 22:27:29 [368353.530119] [<ffffffff81084200>] wait_on_page_writeback+0x16/0x18 Jun 26 22:27:29 [368353.530119] [<ffffffff8108443a>] grab_cache_page_write_begin+0xa9/0xbb Jun 26 22:27:29 [368353.530119] [<ffffffff810d83b6>] block_write_begin+0x27/0x77 Jun 26 22:27:29 [368353.530119] [<ffffffffa0101a00>] ? xfs_get_blocks_direct+0x11/0x11 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0102393>] xfs_vm_write_begin+0x37/0x5b [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810d197a>] ? inode_to_bdi+0x21/0x3f Jun 26 22:27:29 [368353.530119] [<ffffffff8108370b>] generic_file_buffered_write+0x105/0x255 Jun 26 22:27:29 [368353.530119] [<ffffffff810cb70c>] ? mnt_drop_write_file+0xd/0xf Jun 26 22:27:29 [368353.530119] [<ffffffff810c89d8>] ? file_update_time+0xd4/0xeb Jun 26 22:27:29 [368353.530119] [<ffffffffa010677b>] xfs_file_buffered_aio_write+0xb3/0x110 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0106934>] xfs_file_aio_write+0x15c/0x1b0 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8109e1ac>] ? vma_link+0x85/0xa7 Jun 26 22:27:29 [368353.530119] [<ffffffff810b5826>] do_sync_write+0xc9/0x106 Jun 26 22:27:29 [368353.530119] [<ffffffff810b5f3b>] vfs_write+0xa7/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffff810b6133>] sys_write+0x45/0x69 Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] flush-254:11 D ffff88010c8b9980 0 7345 2 0x00000000 Jun 26 22:27:29 [368353.530119] ffff8801004a1640 0000000000000046 ffff8801004a15d0 ffff88010c3cd2e0 Jun 26 22:27:29 [368353.530119] ffff88010f0029c0 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010c8b9980 00000000000107c0 ffff8801004a1fd8 ffff8801004a1fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffffa00b38ff>] get_active_stripe+0x256/0x564 [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:29 [368353.530119] [<ffffffffa00b5976>] make_request+0x1dd/0x3bc [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff8103ffc0>] ? wake_up_bit+0x25/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffffa0069ef1>] md_make_request+0xd0/0x1c2 [md_mod] Jun 26 22:27:29 [368353.530119] [<ffffffff811c6b93>] ? dm_request+0x1b4/0x1c3 Jun 26 22:27:29 [368353.530119] [<ffffffff811c9c8c>] ? linear_merge+0x4d/0x58 Jun 26 22:27:29 [368353.530119] [<ffffffff81119ad1>] generic_make_request+0x97/0xd9 Jun 26 22:27:29 [368353.530119] [<ffffffff81119bee>] submit_bio+0xdb/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffffa01023e5>] xfs_submit_ioend_bio.clone.19+0x2e/0x30 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa01024b5>] xfs_submit_ioend+0xce/0xee [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0102859>] xfs_vm_writepage+0x384/0x412 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108a23e>] __writepage+0x12/0x2b Jun 26 22:27:29 [368353.530119] [<ffffffff8108aa5c>] write_cache_pages+0x225/0x377 Jun 26 22:27:29 [368353.530119] [<ffffffff8108a22c>] ? set_page_dirty+0x5b/0x5b Jun 26 22:27:29 [368353.530119] [<ffffffffa0067408>] ? plugger_unplug+0x38/0x3c [md_mod] Jun 26 22:27:29 [368353.530119] [<ffffffff8111ba86>] ? blk_flush_plug_list+0x94/0x1b1 Jun 26 22:27:29 [368353.530119] [<ffffffff8108abee>] generic_writepages+0x40/0x57 Jun 26 22:27:29 [368353.530119] [<ffffffffa0101c06>] xfs_vm_writepages+0x47/0x4e [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108bbec>] do_writepages+0x1c/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffff810d1c17>] writeback_single_inode+0xcf/0x232 Jun 26 22:27:29 [368353.530119] [<ffffffff810d2015>] writeback_sb_inodes+0x163/0x1fe Jun 26 22:27:29 [368353.530119] [<ffffffff810d240e>] __writeback_inodes_wb+0x6d/0xab Jun 26 22:27:29 [368353.530119] [<ffffffff810d26dd>] wb_writeback+0xfc/0x182 Jun 26 22:27:29 [368353.530119] [<ffffffff810c91b9>] ? get_nr_inodes_unused+0x43/0x5a Jun 26 22:27:29 [368353.530119] [<ffffffff8125f1d9>] ? _raw_spin_lock_irqsave+0xd/0x27 Jun 26 22:27:29 [368353.530119] [<ffffffff810d2dbd>] wb_do_writeback+0x111/0x16d Jun 26 22:27:29 [368353.530119] [<ffffffff81033322>] ? run_timer_softirq+0x1ef/0x1ef Jun 26 22:27:29 [368353.530119] [<ffffffff810d2e9e>] bdi_writeback_thread+0x85/0x14a Jun 26 22:27:29 [368353.530119] [<ffffffff810d2e19>] ? wb_do_writeback+0x16d/0x16d Jun 26 22:27:29 [368353.530119] [<ffffffff810d2e19>] ? wb_do_writeback+0x16d/0x16d Jun 26 22:27:29 [368353.530119] [<ffffffff8103fba9>] kthread+0x84/0x8c Jun 26 22:27:29 [368353.530119] [<ffffffff81260b54>] kernel_thread_helper+0x4/0x10 Jun 26 22:27:29 [368353.530119] [<ffffffff8103fb25>] ? kthread_freezable_should_stop+0x41/0x41 Jun 26 22:27:29 [368353.530119] [<ffffffff81260b50>] ? gs_change+0xb/0xb Jun 26 22:27:29 [368353.530119] irssi D ffffffff8108402d 0 7382 1 0x00000000 Jun 26 22:27:29 [368353.530119] ffff88007f8e3a38 0000000000000082 0000000000000004 ffff88010dd1bfc0 Jun 26 22:27:29 [368353.530119] ffff88007f8e3a08 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010c8b8660 00000000000107c0 ffff88007f8e3fd8 ffff88007f8e3fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8108402d>] ? lock_page+0x22/0x22 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e81e>] io_schedule+0x5e/0x79 Jun 26 22:27:29 [368353.530119] [<ffffffff81084036>] sleep_on_page+0x9/0xd Jun 26 22:27:29 [368353.530119] [<ffffffff8125d8cb>] __wait_on_bit+0x43/0x76 Jun 26 22:27:29 [368353.530119] [<ffffffffa0140b02>] ? xfs_trans_free_item_desc+0x2e/0x30 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810841e3>] wait_on_page_bit+0x6d/0x74 Jun 26 22:27:29 [368353.530119] [<ffffffff8103fff4>] ? autoremove_wake_function+0x34/0x34 Jun 26 22:27:29 [368353.530119] [<ffffffff81084109>] ? find_lock_page+0x32/0x56 Jun 26 22:27:29 [368353.530119] [<ffffffff81084200>] wait_on_page_writeback+0x16/0x18 Jun 26 22:27:29 [368353.530119] [<ffffffff8108443a>] grab_cache_page_write_begin+0xa9/0xbb Jun 26 22:27:29 [368353.530119] [<ffffffff810d83b6>] block_write_begin+0x27/0x77 Jun 26 22:27:29 [368353.530119] [<ffffffffa0101a00>] ? xfs_get_blocks_direct+0x11/0x11 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0102393>] xfs_vm_write_begin+0x37/0x5b [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810d197a>] ? inode_to_bdi+0x21/0x3f Jun 26 22:27:29 [368353.530119] [<ffffffff8108370b>] generic_file_buffered_write+0x105/0x255 Jun 26 22:27:29 [368353.530119] [<ffffffff810cb70c>] ? mnt_drop_write_file+0xd/0xf Jun 26 22:27:29 [368353.530119] [<ffffffff810c89d8>] ? file_update_time+0xd4/0xeb Jun 26 22:27:29 [368353.530119] [<ffffffffa010677b>] xfs_file_buffered_aio_write+0xb3/0x110 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0106934>] xfs_file_aio_write+0x15c/0x1b0 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810b5826>] do_sync_write+0xc9/0x106 Jun 26 22:27:29 [368353.530119] [<ffffffff810b5f3b>] vfs_write+0xa7/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffff810b6133>] sys_write+0x45/0x69 Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] irssi D ffffffff8108402d 0 7383 1 0x00000000 Jun 26 22:27:29 [368353.530119] ffff88007f9f9a38 0000000000000082 0000000000000004 ffff88010dd1f2c0 Jun 26 22:27:29 [368353.530119] ffff88007f9f9a08 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010db4aca0 00000000000107c0 ffff88007f9f9fd8 ffff88007f9f9fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8108402d>] ? lock_page+0x22/0x22 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e81e>] io_schedule+0x5e/0x79 Jun 26 22:27:29 [368353.530119] [<ffffffff81084036>] sleep_on_page+0x9/0xd Jun 26 22:27:29 [368353.530119] [<ffffffff8125d8cb>] __wait_on_bit+0x43/0x76 Jun 26 22:27:29 [368353.530119] [<ffffffffa0140b02>] ? xfs_trans_free_item_desc+0x2e/0x30 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810841e3>] wait_on_page_bit+0x6d/0x74 Jun 26 22:27:29 [368353.530119] [<ffffffff8103fff4>] ? autoremove_wake_function+0x34/0x34 Jun 26 22:27:29 [368353.530119] [<ffffffff81084109>] ? find_lock_page+0x32/0x56 Jun 26 22:27:29 [368353.530119] [<ffffffff81084200>] wait_on_page_writeback+0x16/0x18 Jun 26 22:27:29 [368353.530119] [<ffffffff8108443a>] grab_cache_page_write_begin+0xa9/0xbb Jun 26 22:27:29 [368353.530119] [<ffffffff810d83b6>] block_write_begin+0x27/0x77 Jun 26 22:27:29 [368353.530119] [<ffffffffa0101a00>] ? xfs_get_blocks_direct+0x11/0x11 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0102393>] xfs_vm_write_begin+0x37/0x5b [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810d197a>] ? inode_to_bdi+0x21/0x3f Jun 26 22:27:29 [368353.530119] [<ffffffff8108370b>] generic_file_buffered_write+0x105/0x255 Jun 26 22:27:29 [368353.530119] [<ffffffff810cb70c>] ? mnt_drop_write_file+0xd/0xf Jun 26 22:27:29 [368353.530119] [<ffffffff810c89d8>] ? file_update_time+0xd4/0xeb Jun 26 22:27:29 [368353.530119] [<ffffffffa010677b>] xfs_file_buffered_aio_write+0xb3/0x110 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0106934>] xfs_file_aio_write+0x15c/0x1b0 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810b5826>] do_sync_write+0xc9/0x106 Jun 26 22:27:29 [368353.530119] [<ffffffff810b5f3b>] vfs_write+0xa7/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffff810b6133>] sys_write+0x45/0x69 Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] ping.py D ffff88010c5fcc80 0 4347 4337 0x00000004 Jun 26 22:27:29 [368353.530119] ffff88002c791758 0000000000000082 ffff88002c7916e8 ffff88010c3c8cc0 Jun 26 22:27:29 [368353.530119] 0000000000000400 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010c5fcc80 00000000000107c0 ffff88002c791fd8 ffff88002c791fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffffa00b38ff>] get_active_stripe+0x256/0x564 [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:29 [368353.530119] [<ffffffffa00b5976>] make_request+0x1dd/0x3bc [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff8103ffc0>] ? wake_up_bit+0x25/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffffa0069ef1>] md_make_request+0xd0/0x1c2 [md_mod] Jun 26 22:27:29 [368353.530119] [<ffffffff811c6b93>] ? dm_request+0x1b4/0x1c3 Jun 26 22:27:29 [368353.530119] [<ffffffff811c9c8c>] ? linear_merge+0x4d/0x58 Jun 26 22:27:29 [368353.530119] [<ffffffff81119ad1>] generic_make_request+0x97/0xd9 Jun 26 22:27:29 [368353.530119] [<ffffffff81119bee>] submit_bio+0xdb/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffffa01023e5>] xfs_submit_ioend_bio.clone.19+0x2e/0x30 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa01024b5>] xfs_submit_ioend+0xce/0xee [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0102859>] xfs_vm_writepage+0x384/0x412 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108a23e>] __writepage+0x12/0x2b Jun 26 22:27:29 [368353.530119] [<ffffffff8108aa5c>] write_cache_pages+0x225/0x377 Jun 26 22:27:29 [368353.530119] [<ffffffff8108a22c>] ? set_page_dirty+0x5b/0x5b Jun 26 22:27:29 [368353.530119] [<ffffffffa0102338>] ? xfs_vm_write_end+0x22/0x46 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810837e2>] ? generic_file_buffered_write+0x1dc/0x255 Jun 26 22:27:29 [368353.530119] [<ffffffff810cb70c>] ? mnt_drop_write_file+0xd/0xf Jun 26 22:27:29 [368353.530119] [<ffffffff8108abee>] generic_writepages+0x40/0x57 Jun 26 22:27:29 [368353.530119] [<ffffffffa0101c06>] xfs_vm_writepages+0x47/0x4e [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108bbec>] do_writepages+0x1c/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffff81084a71>] __filemap_fdatawrite_range+0x4e/0x50 Jun 26 22:27:29 [368353.530119] [<ffffffff810855e3>] filemap_fdatawrite_range+0xe/0x10 Jun 26 22:27:29 [368353.530119] [<ffffffffa010822a>] xfs_flush_pages+0x5c/0x8e [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa010c6b0>] xfs_setattr_size+0x135/0x348 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8102e526>] ? current_fs_time+0x22/0x29 Jun 26 22:27:29 [368353.530119] [<ffffffffa010c8de>] xfs_vn_setattr+0x1b/0x2f [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810ca449>] notify_change+0x1a1/0x27a Jun 26 22:27:29 [368353.530119] [<ffffffff810b4bac>] do_truncate+0x65/0x82 Jun 26 22:27:29 [368353.530119] [<ffffffff810b5707>] ? fsnotify_modify+0x5d/0x65 Jun 26 22:27:29 [368353.530119] [<ffffffff810b6dba>] ? fget+0x37/0x4c Jun 26 22:27:29 [368353.530119] [<ffffffff810b4d7b>] sys_ftruncate+0xa4/0xa6 Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] kworker/3:3 D ffff880113d907c0 0 29654 2 0x00000000 Jun 26 22:27:29 [368353.530119] ffff880106145cd0 0000000000000046 00000e3100000000 ffff880072938660 Jun 26 22:27:29 [368353.530119] 0000000081049200 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010b2d3960 00000000000107c0 ffff880106145fd8 ffff880106145fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffffa014210f>] xlog_wait+0x58/0x70 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:29 [368353.530119] [<ffffffffa01442a9>] _xfs_log_force_lsn+0x1bf/0x1f5 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0141a69>] xfs_trans_commit+0xdc/0x1d8 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa011007b>] ? xfs_ifunlock+0x3f/0x3f [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0108011>] xfs_fs_log_dummy+0x6f/0x76 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0143482>] ? xfs_log_need_covered+0x52/0x94 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa01100b9>] xfs_sync_worker+0x3e/0x64 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8103bbd0>] process_one_work+0x161/0x22b Jun 26 22:27:29 [368353.530119] [<ffffffff8103c724>] worker_thread+0xfe/0x182 Jun 26 22:27:29 [368353.530119] [<ffffffff8103c626>] ? manage_workers.clone.22+0x16d/0x16d Jun 26 22:27:29 [368353.530119] [<ffffffff8103c626>] ? manage_workers.clone.22+0x16d/0x16d Jun 26 22:27:29 [368353.530119] [<ffffffff8103fba9>] kthread+0x84/0x8c Jun 26 22:27:29 [368353.530119] [<ffffffff81260b54>] kernel_thread_helper+0x4/0x10 Jun 26 22:27:29 [368353.530119] [<ffffffff8103fb25>] ? kthread_freezable_should_stop+0x41/0x41 Jun 26 22:27:29 [368353.530119] [<ffffffff81260b50>] ? gs_change+0xb/0xb Jun 26 22:27:29 [368353.530119] kworker/0:0 D ffff88010cfa6c60 0 1339 2 0x00000000 Jun 26 22:27:29 [368353.530119] ffff880035d53cd0 0000000000000046 0000023100000000 ffff88010c3c8cc0 Jun 26 22:27:29 [368353.530119] 0000000081049200 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010cfa6c60 00000000000107c0 ffff880035d53fd8 ffff880035d53fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffffa014210f>] xlog_wait+0x58/0x70 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:29 [368353.530119] [<ffffffffa01442a9>] _xfs_log_force_lsn+0x1bf/0x1f5 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0141a69>] xfs_trans_commit+0xdc/0x1d8 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa011007b>] ? xfs_ifunlock+0x3f/0x3f [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0108011>] xfs_fs_log_dummy+0x6f/0x76 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0143493>] ? xfs_log_need_covered+0x63/0x94 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa01100b9>] xfs_sync_worker+0x3e/0x64 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8103bbd0>] process_one_work+0x161/0x22b Jun 26 22:27:29 [368353.530119] [<ffffffff8103c724>] worker_thread+0xfe/0x182 Jun 26 22:27:29 [368353.530119] [<ffffffff8103c626>] ? manage_workers.clone.22+0x16d/0x16d Jun 26 22:27:29 [368353.530119] [<ffffffff8103c626>] ? manage_workers.clone.22+0x16d/0x16d Jun 26 22:27:29 [368353.530119] [<ffffffff8103fba9>] kthread+0x84/0x8c Jun 26 22:27:29 [368353.530119] [<ffffffff81260b54>] kernel_thread_helper+0x4/0x10 Jun 26 22:27:29 [368353.530119] [<ffffffff8103fb25>] ? kthread_freezable_should_stop+0x41/0x41 Jun 26 22:27:29 [368353.530119] [<ffffffff81260b50>] ? gs_change+0xb/0xb Jun 26 22:27:29 [368353.530119] kworker/3:2 D ffff88010cfa0000 0 1968 2 0x00000000 Jun 26 22:27:29 [368353.530119] ffff8800e6f57ac0 0000000000000046 ffffea00040b7900 ffff88010c3cec60 Jun 26 22:27:29 [368353.530119] ffff8800e6f57ae0 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010cfa0000 00000000000107c0 ffff8800e6f57fd8 ffff8800e6f57fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffffa00b38ff>] get_active_stripe+0x256/0x564 [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:29 [368353.530119] [<ffffffffa00b5976>] make_request+0x1dd/0x3bc [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffffa00d52a9>] ? ata_scsi_em_message_show+0x2f/0x2f [libata] Jun 26 22:27:29 [368353.530119] [<ffffffff8103ffc0>] ? wake_up_bit+0x25/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffffa0069ef1>] md_make_request+0xd0/0x1c2 [md_mod] Jun 26 22:27:29 [368353.530119] [<ffffffff81119a05>] ? generic_make_request_checks+0x154/0x189 Jun 26 22:27:29 [368353.530119] [<ffffffff810daed8>] ? bio_alloc_bioset+0x71/0xbf Jun 26 22:27:29 [368353.530119] [<ffffffff811c6ba2>] ? dm_request+0x1c3/0x1c3 Jun 26 22:27:29 [368353.530119] [<ffffffff81119ad1>] generic_make_request+0x97/0xd9 Jun 26 22:27:29 [368353.530119] [<ffffffff811c5c7e>] __map_bio+0x4f/0xb0 Jun 26 22:27:29 [368353.530119] [<ffffffff811c67ae>] __split_and_process_bio+0x246/0x477 Jun 26 22:27:29 [368353.530119] [<ffffffff811c5c13>] ? dec_pending+0x24c/0x268 Jun 26 22:27:29 [368353.530119] [<ffffffff811c6ba2>] ? dm_request+0x1c3/0x1c3 Jun 26 22:27:29 [368353.530119] [<ffffffff811c6c33>] dm_wq_work+0x91/0xb2 Jun 26 22:27:29 [368353.530119] [<ffffffff8103bbd0>] process_one_work+0x161/0x22b Jun 26 22:27:29 [368353.530119] [<ffffffff8103c724>] worker_thread+0xfe/0x182 Jun 26 22:27:29 [368353.530119] [<ffffffff8103c626>] ? manage_workers.clone.22+0x16d/0x16d Jun 26 22:27:29 [368353.530119] [<ffffffff8103c626>] ? manage_workers.clone.22+0x16d/0x16d Jun 26 22:27:29 [368353.530119] [<ffffffff8103fba9>] kthread+0x84/0x8c Jun 26 22:27:29 [368353.530119] [<ffffffff81260b54>] kernel_thread_helper+0x4/0x10 Jun 26 22:27:29 [368353.530119] [<ffffffff8103fb25>] ? kthread_freezable_should_stop+0x41/0x41 Jun 26 22:27:29 [368353.530119] [<ffffffff81260b50>] ? gs_change+0xb/0xb Jun 26 22:27:29 [368353.530119] cleanup D ffff88010de5aca0 0 7088 7286 0x00000000 Jun 26 22:27:29 [368353.530119] ffff880038865868 0000000000000082 ffff8800388657f8 ffffffff81359020 Jun 26 22:27:29 [368353.530119] 0000000000000005 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010de5aca0 00000000000107c0 ffff880038865fd8 ffff880038865fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffffa00b38ff>] get_active_stripe+0x256/0x564 [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:29 [368353.530119] [<ffffffffa00b5976>] make_request+0x1dd/0x3bc [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff8103ffc0>] ? wake_up_bit+0x25/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffffa0069ef1>] md_make_request+0xd0/0x1c2 [md_mod] Jun 26 22:27:29 [368353.530119] [<ffffffff811c6b93>] ? dm_request+0x1b4/0x1c3 Jun 26 22:27:29 [368353.530119] [<ffffffff811c9c8c>] ? linear_merge+0x4d/0x58 Jun 26 22:27:29 [368353.530119] [<ffffffff81119ad1>] generic_make_request+0x97/0xd9 Jun 26 22:27:29 [368353.530119] [<ffffffff81119bee>] submit_bio+0xdb/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffffa01023e5>] xfs_submit_ioend_bio.clone.19+0x2e/0x30 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa01024b5>] xfs_submit_ioend+0xce/0xee [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0102859>] xfs_vm_writepage+0x384/0x412 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108a23e>] __writepage+0x12/0x2b Jun 26 22:27:29 [368353.530119] [<ffffffff8108aa5c>] write_cache_pages+0x225/0x377 Jun 26 22:27:29 [368353.530119] [<ffffffff8108a22c>] ? set_page_dirty+0x5b/0x5b Jun 26 22:27:29 [368353.530119] [<ffffffffa014d43a>] ? xfs_get_acl+0x23/0x104 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa014d669>] ? xfs_acl_chmod+0x2c/0x7b [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108abee>] generic_writepages+0x40/0x57 Jun 26 22:27:29 [368353.530119] [<ffffffffa0101c06>] xfs_vm_writepages+0x47/0x4e [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108bbec>] do_writepages+0x1c/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffff81084a71>] __filemap_fdatawrite_range+0x4e/0x50 Jun 26 22:27:29 [368353.530119] [<ffffffff81084a9b>] filemap_write_and_wait_range+0x28/0x51 Jun 26 22:27:29 [368353.530119] [<ffffffffa0105d97>] xfs_file_fsync+0x3c/0x149 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810d589e>] vfs_fsync_range+0x1e/0x20 Jun 26 22:27:29 [368353.530119] [<ffffffff810d58b7>] vfs_fsync+0x17/0x19 Jun 26 22:27:29 [368353.530119] [<ffffffff810d5923>] do_fsync+0x29/0x3e Jun 26 22:27:29 [368353.530119] [<ffffffff810d5b52>] sys_fsync+0xb/0xf Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] deliver D ffff88007293aca0 0 7095 7094 0x00000004 Jun 26 22:27:29 [368353.530119] ffff880035d21868 0000000000000086 ffff880035d217f8 ffff88010dd1d940 Jun 26 22:27:29 [368353.530119] 0000000000000005 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88007293aca0 00000000000107c0 ffff880035d21fd8 ffff880035d21fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffffa00b38ff>] get_active_stripe+0x256/0x564 [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:29 [368353.530119] [<ffffffffa00b5976>] make_request+0x1dd/0x3bc [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff8103ffc0>] ? wake_up_bit+0x25/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffffa0069ef1>] md_make_request+0xd0/0x1c2 [md_mod] Jun 26 22:27:29 [368353.530119] [<ffffffff811c6b93>] ? dm_request+0x1b4/0x1c3 Jun 26 22:27:29 [368353.530119] [<ffffffff811c9c8c>] ? linear_merge+0x4d/0x58 Jun 26 22:27:29 [368353.530119] [<ffffffff81119ad1>] generic_make_request+0x97/0xd9 Jun 26 22:27:29 [368353.530119] [<ffffffff81119bee>] submit_bio+0xdb/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffffa01023e5>] xfs_submit_ioend_bio.clone.19+0x2e/0x30 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa01024b5>] xfs_submit_ioend+0xce/0xee [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0102859>] xfs_vm_writepage+0x384/0x412 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108a23e>] __writepage+0x12/0x2b Jun 26 22:27:29 [368353.530119] [<ffffffff8108aa5c>] write_cache_pages+0x225/0x377 Jun 26 22:27:29 [368353.530119] [<ffffffff8108a22c>] ? set_page_dirty+0x5b/0x5b Jun 26 22:27:29 [368353.530119] [<ffffffff8109afb4>] ? handle_pte_fault+0x25d/0x615 Jun 26 22:27:29 [368353.530119] [<ffffffffa010c517>] ? xfs_setattr_nonsize+0x433/0x497 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108abee>] generic_writepages+0x40/0x57 Jun 26 22:27:29 [368353.530119] [<ffffffffa0101c06>] xfs_vm_writepages+0x47/0x4e [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108bbec>] do_writepages+0x1c/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffff81084a71>] __filemap_fdatawrite_range+0x4e/0x50 Jun 26 22:27:29 [368353.530119] [<ffffffff81084a9b>] filemap_write_and_wait_range+0x28/0x51 Jun 26 22:27:29 [368353.530119] [<ffffffffa0105d97>] xfs_file_fsync+0x3c/0x149 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810d589e>] vfs_fsync_range+0x1e/0x20 Jun 26 22:27:29 [368353.530119] [<ffffffff810d58b7>] vfs_fsync+0x17/0x19 Jun 26 22:27:29 [368353.530119] [<ffffffff810d5923>] do_fsync+0x29/0x3e Jun 26 22:27:29 [368353.530119] [<ffffffff810d5b52>] sys_fsync+0xb/0xf Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] fetchmail D ffff88010ca1ec60 0 7104 7099 0x00000000 Jun 26 22:27:29 [368353.530119] ffff8800388b36d8 0000000000000086 ffff88007293dfe8 ffff88010b2d4620 Jun 26 22:27:29 [368353.530119] ffff8800388b3678 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010ca1ec60 00000000000107c0 ffff8800388b3fd8 ffff8800388b3fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125f1d9>] ? _raw_spin_lock_irqsave+0xd/0x27 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffff8125d686>] schedule_timeout+0x22/0xd5 Jun 26 22:27:29 [368353.530119] [<ffffffffa0114bd2>] ? kmem_zone_alloc+0x38/0x82 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8125defb>] __down_common+0x90/0xe5 Jun 26 22:27:29 [368353.530119] [<ffffffff8125dfaf>] __down+0x18/0x1a Jun 26 22:27:29 [368353.530119] [<ffffffff81043ef9>] down+0x28/0x38 Jun 26 22:27:29 [368353.530119] [<ffffffffa010399f>] xfs_buf_lock+0x37/0x3b [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0103b1b>] _xfs_buf_find+0x178/0x1d6 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0103d0d>] xfs_buf_get+0x25/0x13b [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0104019>] xfs_buf_read+0x13/0x80 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa01478b9>] xfs_trans_read_buf+0x181/0x248 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa012b2ec>] xfs_da_do_buf+0x2fe/0x518 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa012b558>] xfs_da_read_buf+0x1c/0x1e [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8103feed>] ? bit_waitqueue+0x12/0x92 Jun 26 22:27:29 [368353.530119] [<ffffffffa01309ed>] xfs_dir2_leaf_addname+0x43/0x59e [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0121665>] ? xfs_bmap_last_extent.clone.23+0x62/0x75 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa012d714>] xfs_dir_createname+0x14f/0x16b [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0113723>] xfs_create+0x31e/0x4a6 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa010bf55>] xfs_vn_mknod+0xd1/0x153 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa010bff8>] xfs_vn_create+0xe/0x10 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810c05a5>] vfs_create+0x52/0x72 Jun 26 22:27:29 [368353.530119] [<ffffffff810c0d82>] do_last+0x220/0x573 Jun 26 22:27:29 [368353.530119] [<ffffffff810c17c3>] path_openat+0xcb/0x319 Jun 26 22:27:29 [368353.530119] [<ffffffff810c1aff>] do_filp_open+0x38/0x84 Jun 26 22:27:29 [368353.530119] [<ffffffff810cb079>] ? alloc_fd+0x6f/0xfe Jun 26 22:27:29 [368353.530119] [<ffffffff810b54bf>] do_sys_open+0x70/0x102 Jun 26 22:27:29 [368353.530119] [<ffffffff810b556d>] sys_open+0x1c/0x1e Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] munin-update D ffffffff8108402d 0 7113 7105 0x00000000 Jun 26 22:27:29 [368353.530119] ffff88005ea15a38 0000000000000082 0000000000000018 ffffffff81359020 Jun 26 22:27:29 [368353.530119] ffff88005ea15a28 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010de59320 00000000000107c0 ffff88005ea15fd8 ffff88005ea15fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8108402d>] ? lock_page+0x22/0x22 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e81e>] io_schedule+0x5e/0x79 Jun 26 22:27:29 [368353.530119] [<ffffffff81084036>] sleep_on_page+0x9/0xd Jun 26 22:27:29 [368353.530119] [<ffffffff8125d8cb>] __wait_on_bit+0x43/0x76 Jun 26 22:27:29 [368353.530119] [<ffffffffa0140b02>] ? xfs_trans_free_item_desc+0x2e/0x30 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810841e3>] wait_on_page_bit+0x6d/0x74 Jun 26 22:27:29 [368353.530119] [<ffffffff8103fff4>] ? autoremove_wake_function+0x34/0x34 Jun 26 22:27:29 [368353.530119] [<ffffffff81084109>] ? find_lock_page+0x32/0x56 Jun 26 22:27:29 [368353.530119] [<ffffffff81084200>] wait_on_page_writeback+0x16/0x18 Jun 26 22:27:29 [368353.530119] [<ffffffff8108443a>] grab_cache_page_write_begin+0xa9/0xbb Jun 26 22:27:29 [368353.530119] [<ffffffff810d83b6>] block_write_begin+0x27/0x77 Jun 26 22:27:29 [368353.530119] [<ffffffffa0101a00>] ? xfs_get_blocks_direct+0x11/0x11 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0102393>] xfs_vm_write_begin+0x37/0x5b [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810d197a>] ? inode_to_bdi+0x21/0x3f Jun 26 22:27:29 [368353.530119] [<ffffffff8108370b>] generic_file_buffered_write+0x105/0x255 Jun 26 22:27:29 [368353.530119] [<ffffffff810cb70c>] ? mnt_drop_write_file+0xd/0xf Jun 26 22:27:29 [368353.530119] [<ffffffff810c89d8>] ? file_update_time+0xd4/0xeb Jun 26 22:27:29 [368353.530119] [<ffffffffa010677b>] xfs_file_buffered_aio_write+0xb3/0x110 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0106934>] xfs_file_aio_write+0x15c/0x1b0 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810b5826>] do_sync_write+0xc9/0x106 Jun 26 22:27:29 [368353.530119] [<ffffffff8109b5a5>] ? handle_mm_fault+0xe1/0xf6 Jun 26 22:27:29 [368353.530119] [<ffffffff810b5f3b>] vfs_write+0xa7/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffff810b6133>] sys_write+0x45/0x69 Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] fetchmail D 0000000000000002 0 7144 7138 0x00000000 Jun 26 22:27:29 [368353.530119] ffff88007f8cbc78 0000000000000082 ffff88010ede7b40 ffffffff81359020 Jun 26 22:27:29 [368353.530119] ffff88007f8cbd48 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88007293f920 00000000000107c0 ffff88007f8cbfd8 ffff88007f8cbfd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e95b>] schedule_preempt_disabled+0x9/0xb Jun 26 22:27:29 [368353.530119] [<ffffffff8125db87>] __mutex_lock_common.clone.5+0x11d/0x17e Jun 26 22:27:29 [368353.530119] [<ffffffff8125dbf6>] __mutex_lock_slowpath+0xe/0x10 Jun 26 22:27:29 [368353.530119] [<ffffffff8125d988>] mutex_lock+0x16/0x27 Jun 26 22:27:29 [368353.530119] [<ffffffff810c0cf7>] do_last+0x195/0x573 Jun 26 22:27:29 [368353.530119] [<ffffffff810c17c3>] path_openat+0xcb/0x319 Jun 26 22:27:29 [368353.530119] [<ffffffff810c1aff>] do_filp_open+0x38/0x84 Jun 26 22:27:29 [368353.530119] [<ffffffff810cb079>] ? alloc_fd+0x6f/0xfe Jun 26 22:27:29 [368353.530119] [<ffffffff810b54bf>] do_sys_open+0x70/0x102 Jun 26 22:27:29 [368353.530119] [<ffffffff810b556d>] sys_open+0x1c/0x1e Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] munin-update D 0000000000000002 0 7145 7141 0x00000000 Jun 26 22:27:29 [368353.530119] ffff8800389ddbe8 0000000000000086 ffff8800389ddbd8 ffff88010dd1d940 Jun 26 22:27:29 [368353.530119] 0000000000000002 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010b2d5fa0 00000000000107c0 ffff8800389ddfd8 ffff8800389ddfd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e95b>] schedule_preempt_disabled+0x9/0xb Jun 26 22:27:29 [368353.530119] [<ffffffff8125db87>] __mutex_lock_common.clone.5+0x11d/0x17e Jun 26 22:27:29 [368353.530119] [<ffffffff810858d0>] ? filemap_fault+0x1ee/0x34f Jun 26 22:27:29 [368353.530119] [<ffffffff8125dbf6>] __mutex_lock_slowpath+0xe/0x10 Jun 26 22:27:29 [368353.530119] [<ffffffff8125d988>] mutex_lock+0x16/0x27 Jun 26 22:27:29 [368353.530119] [<ffffffffa0105b7c>] xfs_rw_ilock+0x21/0x33 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0106715>] xfs_file_buffered_aio_write+0x4d/0x110 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0106934>] xfs_file_aio_write+0x15c/0x1b0 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810b5826>] do_sync_write+0xc9/0x106 Jun 26 22:27:29 [368353.530119] [<ffffffff8109b5a5>] ? handle_mm_fault+0xe1/0xf6 Jun 26 22:27:29 [368353.530119] [<ffffffff810b5f3b>] vfs_write+0xa7/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffff810b6133>] sys_write+0x45/0x69 Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] fetchmail D 0000000000000002 0 7181 7175 0x00000000 Jun 26 22:27:29 [368353.530119] ffff880038901c78 0000000000000086 ffff88010ede7b40 ffff88007293c620 Jun 26 22:27:29 [368353.530119] ffff880038901d48 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88007293bfc0 00000000000107c0 ffff880038901fd8 ffff880038901fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e95b>] schedule_preempt_disabled+0x9/0xb Jun 26 22:27:29 [368353.530119] [<ffffffff8125db87>] __mutex_lock_common.clone.5+0x11d/0x17e Jun 26 22:27:29 [368353.530119] [<ffffffff8125dbf6>] __mutex_lock_slowpath+0xe/0x10 Jun 26 22:27:29 [368353.530119] [<ffffffff8125d988>] mutex_lock+0x16/0x27 Jun 26 22:27:29 [368353.530119] [<ffffffff810c0cf7>] do_last+0x195/0x573 Jun 26 22:27:29 [368353.530119] [<ffffffff810c17c3>] path_openat+0xcb/0x319 Jun 26 22:27:29 [368353.530119] [<ffffffff810c1aff>] do_filp_open+0x38/0x84 Jun 26 22:27:29 [368353.530119] [<ffffffff810cb079>] ? alloc_fd+0x6f/0xfe Jun 26 22:27:29 [368353.530119] [<ffffffff810b54bf>] do_sys_open+0x70/0x102 Jun 26 22:27:29 [368353.530119] [<ffffffff810b556d>] sys_open+0x1c/0x1e Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] munin-update D 0000000000000002 0 7189 7183 0x00000000 Jun 26 22:27:29 [368353.530119] ffff880037793be8 0000000000000082 ffff880037793bd8 ffff88010dd1bfc0 Jun 26 22:27:29 [368353.530119] 0000000000000002 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010de59fe0 00000000000107c0 ffff880037793fd8 ffff880037793fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e95b>] schedule_preempt_disabled+0x9/0xb Jun 26 22:27:29 [368353.530119] [<ffffffff8125db87>] __mutex_lock_common.clone.5+0x11d/0x17e Jun 26 22:27:29 [368353.530119] [<ffffffff810858d0>] ? filemap_fault+0x1ee/0x34f Jun 26 22:27:29 [368353.530119] [<ffffffff8125dbf6>] __mutex_lock_slowpath+0xe/0x10 Jun 26 22:27:29 [368353.530119] [<ffffffff8125d988>] mutex_lock+0x16/0x27 Jun 26 22:27:29 [368353.530119] [<ffffffffa0105b7c>] xfs_rw_ilock+0x21/0x33 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0106715>] xfs_file_buffered_aio_write+0x4d/0x110 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0106934>] xfs_file_aio_write+0x15c/0x1b0 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810b5826>] do_sync_write+0xc9/0x106 Jun 26 22:27:29 [368353.530119] [<ffffffff8109b5a5>] ? handle_mm_fault+0xe1/0xf6 Jun 26 22:27:29 [368353.530119] [<ffffffff810b5f3b>] vfs_write+0xa7/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffff810b6133>] sys_write+0x45/0x69 Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] apache2 D ffff88010de5ec60 0 7208 17519 0x00000000 Jun 26 22:27:29 [368353.530119] ffff8800e440d778 0000000000000086 ffff8800e440d708 ffff88010dd1f2c0 Jun 26 22:27:29 [368353.530119] 0000000000000000 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010de5ec60 00000000000107c0 ffff8800e440dfd8 ffff8800e440dfd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffffa00b38ff>] get_active_stripe+0x256/0x564 [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:29 [368353.530119] [<ffffffffa00b5976>] make_request+0x1dd/0x3bc [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff8103ffc0>] ? wake_up_bit+0x25/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffffa0069ef1>] md_make_request+0xd0/0x1c2 [md_mod] Jun 26 22:27:29 [368353.530119] [<ffffffff811c6b93>] ? dm_request+0x1b4/0x1c3 Jun 26 22:27:29 [368353.530119] [<ffffffff81119ad1>] generic_make_request+0x97/0xd9 Jun 26 22:27:29 [368353.530119] [<ffffffff81119bee>] submit_bio+0xdb/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffff8108cb59>] ? __lru_cache_add+0x2f/0x56 Jun 26 22:27:29 [368353.530119] [<ffffffff810df1e1>] mpage_readpages+0x120/0x134 Jun 26 22:27:29 [368353.530119] [<ffffffffa0101a00>] ? xfs_get_blocks_direct+0x11/0x11 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0101a00>] ? xfs_get_blocks_direct+0x11/0x11 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0101bbd>] xfs_vm_readpages+0x18/0x1a [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108be39>] __do_page_cache_readahead+0x135/0x1d2 Jun 26 22:27:29 [368353.530119] [<ffffffff810afc43>] ? signal_pending+0x12/0x1c Jun 26 22:27:29 [368353.530119] [<ffffffff8108c14c>] ra_submit+0x1c/0x20 Jun 26 22:27:29 [368353.530119] [<ffffffff8108c307>] ondemand_readahead+0x1b7/0x1c6 Jun 26 22:27:29 [368353.530119] [<ffffffff8108c3cb>] page_cache_sync_readahead+0x38/0x3d Jun 26 22:27:29 [368353.530119] [<ffffffff810851ec>] generic_file_aio_read+0x29d/0x61e Jun 26 22:27:29 [368353.530119] [<ffffffff810a239d>] ? page_add_new_anon_rmap+0x5b/0x6e Jun 26 22:27:29 [368353.530119] [<ffffffffa01062a1>] xfs_file_aio_read+0x1a4/0x1fa [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810b592c>] do_sync_read+0xc9/0x106 Jun 26 22:27:29 [368353.530119] [<ffffffff8109b5a5>] ? handle_mm_fault+0xe1/0xf6 Jun 26 22:27:29 [368353.530119] [<ffffffff810b6032>] vfs_read+0xa4/0xf7 Jun 26 22:27:29 [368353.530119] [<ffffffff810b60ca>] sys_read+0x45/0x69 Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] cleanup D ffff88010bd6e600 0 7210 7286 0x00000000 Jun 26 22:27:29 [368353.530119] ffff88003ac71868 0000000000000082 ffff88003ac717f8 ffff88010dd1bfc0 Jun 26 22:27:29 [368353.530119] 0000000000000005 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88010bd6e600 00000000000107c0 ffff88003ac71fd8 ffff88003ac71fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffffa00b38ff>] get_active_stripe+0x256/0x564 [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:29 [368353.530119] [<ffffffffa00b5976>] make_request+0x1dd/0x3bc [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff8103ffc0>] ? wake_up_bit+0x25/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffffa0069ef1>] md_make_request+0xd0/0x1c2 [md_mod] Jun 26 22:27:29 [368353.530119] [<ffffffff811c6b93>] ? dm_request+0x1b4/0x1c3 Jun 26 22:27:29 [368353.530119] [<ffffffff811c9c8c>] ? linear_merge+0x4d/0x58 Jun 26 22:27:29 [368353.530119] [<ffffffff81119ad1>] generic_make_request+0x97/0xd9 Jun 26 22:27:29 [368353.530119] [<ffffffff81119bee>] submit_bio+0xdb/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffffa01023e5>] xfs_submit_ioend_bio.clone.19+0x2e/0x30 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa01024b5>] xfs_submit_ioend+0xce/0xee [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0102859>] xfs_vm_writepage+0x384/0x412 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108a23e>] __writepage+0x12/0x2b Jun 26 22:27:29 [368353.530119] [<ffffffff8108aa5c>] write_cache_pages+0x225/0x377 Jun 26 22:27:29 [368353.530119] [<ffffffff8108a22c>] ? set_page_dirty+0x5b/0x5b Jun 26 22:27:29 [368353.530119] [<ffffffffa014d669>] ? xfs_acl_chmod+0x2c/0x7b [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108abee>] generic_writepages+0x40/0x57 Jun 26 22:27:29 [368353.530119] [<ffffffffa0101c06>] xfs_vm_writepages+0x47/0x4e [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108bbec>] do_writepages+0x1c/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffff81084a71>] __filemap_fdatawrite_range+0x4e/0x50 Jun 26 22:27:29 [368353.530119] [<ffffffff81084a9b>] filemap_write_and_wait_range+0x28/0x51 Jun 26 22:27:29 [368353.530119] [<ffffffffa0105d97>] xfs_file_fsync+0x3c/0x149 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810d589e>] vfs_fsync_range+0x1e/0x20 Jun 26 22:27:29 [368353.530119] [<ffffffff810d58b7>] vfs_fsync+0x17/0x19 Jun 26 22:27:29 [368353.530119] [<ffffffff810d5923>] do_fsync+0x29/0x3e Jun 26 22:27:29 [368353.530119] [<ffffffff810d5b52>] sys_fsync+0xb/0xf Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] local D ffffffff8108402d 0 7213 7286 0x00000000 Jun 26 22:27:29 [368353.530119] ffff8800e6e17a38 0000000000000086 ffff8800e6e179b8 ffff88010dd1d940 Jun 26 22:27:29 [368353.530119] ffff88010f002240 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff880072938cc0 00000000000107c0 ffff8800e6e17fd8 ffff8800e6e17fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8108402d>] ? lock_page+0x22/0x22 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffff8125e81e>] io_schedule+0x5e/0x79 Jun 26 22:27:29 [368353.530119] [<ffffffff81084036>] sleep_on_page+0x9/0xd Jun 26 22:27:29 [368353.530119] [<ffffffff8125d8cb>] __wait_on_bit+0x43/0x76 Jun 26 22:27:29 [368353.530119] [<ffffffffa0140b02>] ? xfs_trans_free_item_desc+0x2e/0x30 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810841e3>] wait_on_page_bit+0x6d/0x74 Jun 26 22:27:29 [368353.530119] [<ffffffff8103fff4>] ? autoremove_wake_function+0x34/0x34 Jun 26 22:27:29 [368353.530119] [<ffffffff81084109>] ? find_lock_page+0x32/0x56 Jun 26 22:27:29 [368353.530119] [<ffffffff81084200>] wait_on_page_writeback+0x16/0x18 Jun 26 22:27:29 [368353.530119] [<ffffffff8108443a>] grab_cache_page_write_begin+0xa9/0xbb Jun 26 22:27:29 [368353.530119] [<ffffffff810d83b6>] block_write_begin+0x27/0x77 Jun 26 22:27:29 [368353.530119] [<ffffffffa0101a00>] ? xfs_get_blocks_direct+0x11/0x11 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0102393>] xfs_vm_write_begin+0x37/0x5b [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff8108370b>] generic_file_buffered_write+0x105/0x255 Jun 26 22:27:29 [368353.530119] [<ffffffff810cb70c>] ? mnt_drop_write_file+0xd/0xf Jun 26 22:27:29 [368353.530119] [<ffffffff810c89d8>] ? file_update_time+0xd4/0xeb Jun 26 22:27:29 [368353.530119] [<ffffffffa010677b>] xfs_file_buffered_aio_write+0xb3/0x110 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0106934>] xfs_file_aio_write+0x15c/0x1b0 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffff810b5826>] do_sync_write+0xc9/0x106 Jun 26 22:27:29 [368353.530119] [<ffffffff8109b5a5>] ? handle_mm_fault+0xe1/0xf6 Jun 26 22:27:29 [368353.530119] [<ffffffff810b5f3b>] vfs_write+0xa7/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffff810b6133>] sys_write+0x45/0x69 Jun 26 22:27:29 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:29 [368353.530119] sshd D ffff88007293d2e0 0 7221 1 0x00000000 Jun 26 22:27:29 [368353.530119] ffff880035da5868 0000000000000082 ffff880035da57f8 ffff88010dd1d940 Jun 26 22:27:29 [368353.530119] ffff88010f0029c0 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:29 [368353.530119] ffff88007293d2e0 00000000000107c0 ffff880035da5fd8 ffff880035da5fd8 Jun 26 22:27:29 [368353.530119] Call Trace: Jun 26 22:27:29 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:29 [368353.530119] [<ffffffffa00b38ff>] get_active_stripe+0x256/0x564 [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:29 [368353.530119] [<ffffffffa00b5976>] make_request+0x1dd/0x3bc [raid456] Jun 26 22:27:29 [368353.530119] [<ffffffff8103ffc0>] ? wake_up_bit+0x25/0x25 Jun 26 22:27:29 [368353.530119] [<ffffffffa0069ef1>] md_make_request+0xd0/0x1c2 [md_mod] Jun 26 22:27:29 [368353.530119] [<ffffffff811c6b93>] ? dm_request+0x1b4/0x1c3 Jun 26 22:27:29 [368353.530119] [<ffffffff811c9c8c>] ? linear_merge+0x4d/0x58 Jun 26 22:27:29 [368353.530119] [<ffffffff81119ad1>] generic_make_request+0x97/0xd9 Jun 26 22:27:29 [368353.530119] [<ffffffff81119bee>] submit_bio+0xdb/0xfa Jun 26 22:27:29 [368353.530119] [<ffffffffa01023e5>] xfs_submit_ioend_bio.clone.19+0x2e/0x30 [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa01024b5>] xfs_submit_ioend+0xce/0xee [xfs] Jun 26 22:27:29 [368353.530119] [<ffffffffa0102859>] xfs_vm_writepage+0x384/0x412 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffff8108a23e>] __writepage+0x12/0x2b Jun 26 22:27:30 [368353.530119] [<ffffffff8108aa5c>] write_cache_pages+0x225/0x377 Jun 26 22:27:30 [368353.530119] [<ffffffff8108a22c>] ? set_page_dirty+0x5b/0x5b Jun 26 22:27:30 [368353.530119] [<ffffffffa010696b>] ? xfs_file_aio_write+0x193/0x1b0 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffff8108abee>] generic_writepages+0x40/0x57 Jun 26 22:27:30 [368353.530119] [<ffffffffa0101c06>] xfs_vm_writepages+0x47/0x4e [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffff8108bbec>] do_writepages+0x1c/0x25 Jun 26 22:27:30 [368353.530119] [<ffffffff81084a71>] __filemap_fdatawrite_range+0x4e/0x50 Jun 26 22:27:30 [368353.530119] [<ffffffff81084a9b>] filemap_write_and_wait_range+0x28/0x51 Jun 26 22:27:30 [368353.530119] [<ffffffffa0105d97>] xfs_file_fsync+0x3c/0x149 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffff810b5707>] ? fsnotify_modify+0x5d/0x65 Jun 26 22:27:30 [368353.530119] [<ffffffff810d589e>] vfs_fsync_range+0x1e/0x20 Jun 26 22:27:30 [368353.530119] [<ffffffff810d58b7>] vfs_fsync+0x17/0x19 Jun 26 22:27:30 [368353.530119] [<ffffffff810d5923>] do_fsync+0x29/0x3e Jun 26 22:27:30 [368353.530119] [<ffffffff810d5b52>] sys_fsync+0xb/0xf Jun 26 22:27:30 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:30 [368353.530119] bash D fffffffeffffffff 0 7226 7225 0x00000000 Jun 26 22:27:30 [368353.530119] ffff880073ebfb70 0000000000000082 4000000000000000 ffffffff81359020 Jun 26 22:27:30 [368353.530119] 0000000000000202 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:30 [368353.530119] ffff88010bd6f920 00000000000107c0 ffff880073ebffd8 ffff880073ebffd8 Jun 26 22:27:30 [368353.530119] Call Trace: Jun 26 22:27:30 [368353.530119] [<ffffffffa0105975>] ? xfs_dir_fsync+0x77/0x77 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:30 [368353.530119] [<ffffffff8125efa0>] rwsem_down_failed_common+0xd7/0x109 Jun 26 22:27:30 [368353.530119] [<ffffffff8125eff9>] rwsem_down_read_failed+0x12/0x14 Jun 26 22:27:30 [368353.530119] [<ffffffff81136a84>] call_rwsem_down_read_failed+0x14/0x30 Jun 26 22:27:30 [368353.530119] [<ffffffff8125de47>] ? down_read+0x12/0x14 Jun 26 22:27:30 [368353.530119] [<ffffffffa0108483>] xfs_ilock+0x59/0x5e [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa01084b8>] xfs_ilock_map_shared+0x30/0x36 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa010599e>] xfs_dir_open+0x29/0x58 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffff810b4899>] __dentry_open.clone.14+0x149/0x263 Jun 26 22:27:30 [368353.530119] [<ffffffff810cbe68>] ? mntget+0x1b/0x21 Jun 26 22:27:30 [368353.530119] [<ffffffff810b5448>] nameidata_to_filp+0x43/0x4a Jun 26 22:27:30 [368353.530119] [<ffffffff810c10c1>] do_last+0x55f/0x573 Jun 26 22:27:30 [368353.530119] [<ffffffff810c17c3>] path_openat+0xcb/0x319 Jun 26 22:27:30 [368353.530119] [<ffffffff810c1aff>] do_filp_open+0x38/0x84 Jun 26 22:27:30 [368353.530119] [<ffffffff810cb079>] ? alloc_fd+0x6f/0xfe Jun 26 22:27:30 [368353.530119] [<ffffffff810b54bf>] do_sys_open+0x70/0x102 Jun 26 22:27:30 [368353.530119] [<ffffffff810b557e>] sys_openat+0xf/0x11 Jun 26 22:27:30 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:30 [368353.530119] fetchmail D 0000000000000002 0 7237 7231 0x00000000 Jun 26 22:27:30 [368353.530119] ffff8800e6fedc78 0000000000000086 ffff88010ede7b40 ffffffff81359020 Jun 26 22:27:30 [368353.530119] ffff8800e6fedd48 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:30 [368353.530119] ffff880072939980 00000000000107c0 ffff8800e6fedfd8 ffff8800e6fedfd8 Jun 26 22:27:30 [368353.530119] Call Trace: Jun 26 22:27:30 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:30 [368353.530119] [<ffffffff8125e95b>] schedule_preempt_disabled+0x9/0xb Jun 26 22:27:30 [368353.530119] [<ffffffff8125db87>] __mutex_lock_common.clone.5+0x11d/0x17e Jun 26 22:27:30 [368353.530119] [<ffffffff8125dbf6>] __mutex_lock_slowpath+0xe/0x10 Jun 26 22:27:30 [368353.530119] [<ffffffff8125d988>] mutex_lock+0x16/0x27 Jun 26 22:27:30 [368353.530119] [<ffffffff810c0cf7>] do_last+0x195/0x573 Jun 26 22:27:30 [368353.530119] [<ffffffff810c17c3>] path_openat+0xcb/0x319 Jun 26 22:27:30 [368353.530119] [<ffffffff810c1aff>] do_filp_open+0x38/0x84 Jun 26 22:27:30 [368353.530119] [<ffffffff810cb079>] ? alloc_fd+0x6f/0xfe Jun 26 22:27:30 [368353.530119] [<ffffffff810b54bf>] do_sys_open+0x70/0x102 Jun 26 22:27:30 [368353.530119] [<ffffffff810b556d>] sys_open+0x1c/0x1e Jun 26 22:27:30 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:30 [368353.530119] munin-update D 0000000000000002 0 7238 7235 0x00000000 Jun 26 22:27:30 [368353.530119] ffff88002c7fbbe8 0000000000000082 ffff88002c7fbbd8 ffff88010dd1d940 Jun 26 22:27:30 [368353.530119] 0000000000000002 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:30 [368353.530119] ffff88010b2d0000 00000000000107c0 ffff88002c7fbfd8 ffff88002c7fbfd8 Jun 26 22:27:30 [368353.530119] Call Trace: Jun 26 22:27:30 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:30 [368353.530119] [<ffffffff8125e95b>] schedule_preempt_disabled+0x9/0xb Jun 26 22:27:30 [368353.530119] [<ffffffff8125db87>] __mutex_lock_common.clone.5+0x11d/0x17e Jun 26 22:27:30 [368353.530119] [<ffffffff810858d0>] ? filemap_fault+0x1ee/0x34f Jun 26 22:27:30 [368353.530119] [<ffffffff8125dbf6>] __mutex_lock_slowpath+0xe/0x10 Jun 26 22:27:30 [368353.530119] [<ffffffff8125d988>] mutex_lock+0x16/0x27 Jun 26 22:27:30 [368353.530119] [<ffffffffa0105b7c>] xfs_rw_ilock+0x21/0x33 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0106715>] xfs_file_buffered_aio_write+0x4d/0x110 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0106934>] xfs_file_aio_write+0x15c/0x1b0 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffff810b5826>] do_sync_write+0xc9/0x106 Jun 26 22:27:30 [368353.530119] [<ffffffff8109b5a5>] ? handle_mm_fault+0xe1/0xf6 Jun 26 22:27:30 [368353.530119] [<ffffffff810b5f3b>] vfs_write+0xa7/0xfa Jun 26 22:27:30 [368353.530119] [<ffffffff810b6133>] sys_write+0x45/0x69 Jun 26 22:27:30 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:30 [368353.530119] ping.py D ffff88010b2d3fc0 0 7248 7247 0x00000000 Jun 26 22:27:30 [368353.530119] ffff880035cf1288 0000000000000082 0000000000000010 ffff88010dd1f2c0 Jun 26 22:27:30 [368353.530119] 0000000000000000 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:30 [368353.530119] ffff88010b2d3fc0 00000000000107c0 ffff880035cf1fd8 ffff880035cf1fd8 Jun 26 22:27:30 [368353.530119] Call Trace: Jun 26 22:27:30 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:30 [368353.530119] [<ffffffffa00b38ff>] get_active_stripe+0x256/0x564 [raid456] Jun 26 22:27:30 [368353.530119] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:30 [368353.530119] [<ffffffffa00b5976>] make_request+0x1dd/0x3bc [raid456] Jun 26 22:27:30 [368353.530119] [<ffffffff8103ffc0>] ? wake_up_bit+0x25/0x25 Jun 26 22:27:30 [368353.530119] [<ffffffffa0069ef1>] md_make_request+0xd0/0x1c2 [md_mod] Jun 26 22:27:30 [368353.530119] [<ffffffff811c6b93>] ? dm_request+0x1b4/0x1c3 Jun 26 22:27:30 [368353.530119] [<ffffffff81119ad1>] generic_make_request+0x97/0xd9 Jun 26 22:27:30 [368353.530119] [<ffffffff81119bee>] submit_bio+0xdb/0xfa Jun 26 22:27:30 [368353.530119] [<ffffffffa0102e32>] _xfs_buf_ioapply+0x157/0x182 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0103f86>] xfs_buf_iorequest+0x31/0x4a [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa01420b1>] xlog_bdstrat+0x3c/0x42 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0142b4b>] xlog_sync+0x1ea/0x2de [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0143583>] xlog_state_release_iclog+0x8f/0x94 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0143fe5>] _xfs_log_force+0xef/0x1be [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa01440c7>] xfs_log_force+0x13/0x36 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0103960>] xfs_buf_trylock+0x40/0x48 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0103a4c>] _xfs_buf_find+0xa9/0x1d6 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0103d0d>] xfs_buf_get+0x25/0x13b [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa01476ae>] xfs_trans_get_buf+0x9b/0xc3 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffff81049200>] ? try_to_wake_up+0x1ae/0x1ae Jun 26 22:27:30 [368353.530119] [<ffffffffa0134403>] xfs_ialloc_inode_init+0xd1/0x18b [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0134a13>] xfs_ialloc_ag_alloc+0x2ef/0x446 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0134bd1>] ? xfs_read_agi+0x67/0xe1 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0117c29>] ? __xfs_alloc_vextent+0x388/0x388 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa013571c>] xfs_dialloc+0x123/0x75a [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0137096>] xfs_ialloc+0x53/0x4e2 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa01427db>] ? xlog_grant_head_check+0x4b/0xbc [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa01119b8>] xfs_dir_ialloc+0x6c/0x238 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa01136bb>] xfs_create+0x2b6/0x4a6 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffff810c5fe9>] ? __d_rehash+0x1e/0x54 Jun 26 22:27:30 [368353.530119] [<ffffffffa010bf55>] xfs_vn_mknod+0xd1/0x153 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa010bff8>] xfs_vn_create+0xe/0x10 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffff810c05a5>] vfs_create+0x52/0x72 Jun 26 22:27:30 [368353.530119] [<ffffffff810c0d82>] do_last+0x220/0x573 Jun 26 22:27:30 [368353.530119] [<ffffffff810c17c3>] path_openat+0xcb/0x319 Jun 26 22:27:30 [368353.530119] [<ffffffff810c1aff>] do_filp_open+0x38/0x84 Jun 26 22:27:30 [368353.530119] [<ffffffff810cb079>] ? alloc_fd+0x6f/0xfe Jun 26 22:27:30 [368353.530119] [<ffffffff810b54bf>] do_sys_open+0x70/0x102 Jun 26 22:27:30 [368353.530119] [<ffffffff810b556d>] sys_open+0x1c/0x1e Jun 26 22:27:30 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b Jun 26 22:27:30 [368353.530119] cleanup D ffff88010b2d1320 0 7250 7286 0x00000000 Jun 26 22:27:30 [368353.530119] ffff88003779f658 0000000000000086 01ff880113c107c0 ffffffff81359020 Jun 26 22:27:30 [368353.530119] ffff880113d0d450 00000000000107c0 00000000000107c0 00000000000107c0 Jun 26 22:27:30 [368353.530119] ffff88010b2d1320 00000000000107c0 ffff88003779ffd8 ffff88003779ffd8 Jun 26 22:27:30 [368353.530119] Call Trace: Jun 26 22:27:30 [368353.530119] [<ffffffff8125e7be>] schedule+0x5f/0x61 Jun 26 22:27:30 [368353.530119] [<ffffffff8125d686>] schedule_timeout+0x22/0xd5 Jun 26 22:27:30 [368353.530119] [<ffffffff8104c1f7>] ? update_group_power+0xd5/0x134 Jun 26 22:27:30 [368353.530119] [<ffffffff8112df54>] ? cpumask_next_and+0x2c/0x39 Jun 26 22:27:30 [368353.530119] [<ffffffff8125defb>] __down_common+0x90/0xe5 Jun 26 22:27:30 [368353.530119] [<ffffffff810afc43>] ? signal_pending+0x12/0x1c Jun 26 22:27:30 [368353.530119] [<ffffffff8125dfaf>] __down+0x18/0x1a Jun 26 22:27:30 [368353.530119] [<ffffffff81043ef9>] down+0x28/0x38 Jun 26 22:27:30 [368353.530119] [<ffffffffa010399f>] xfs_buf_lock+0x37/0x3b [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0103b1b>] _xfs_buf_find+0x178/0x1d6 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0103d0d>] xfs_buf_get+0x25/0x13b [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0104019>] xfs_buf_read+0x13/0x80 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa01478b9>] xfs_trans_read_buf+0x181/0x248 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0134bd1>] xfs_read_agi+0x67/0xe1 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0134c64>] xfs_ialloc_read_agi+0x19/0x66 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa0135564>] xfs_ialloc_ag_select+0x16d/0x202 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa013563c>] xfs_dialloc+0x43/0x75a [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffff8104b47e>] ? select_task_rq_fair+0x320/0x5ab Jun 26 22:27:30 [368353.530119] [<ffffffff810840d2>] ? unlock_page+0x25/0x2a Jun 26 22:27:30 [368353.530119] [<ffffffff8125f132>] ? _raw_read_unlock_bh+0x12/0x14 Jun 26 22:27:30 [368353.530119] [<ffffffffa0137096>] xfs_ialloc+0x53/0x4e2 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa01427db>] ? xlog_grant_head_check+0x4b/0xbc [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa01119b8>] xfs_dir_ialloc+0x6c/0x238 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa01136bb>] xfs_create+0x2b6/0x4a6 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffff810c5fe9>] ? __d_rehash+0x1e/0x54 Jun 26 22:27:30 [368353.530119] [<ffffffffa010bf55>] xfs_vn_mknod+0xd1/0x153 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffffa010bff8>] xfs_vn_create+0xe/0x10 [xfs] Jun 26 22:27:30 [368353.530119] [<ffffffff810c05a5>] vfs_create+0x52/0x72 Jun 26 22:27:30 [368353.530119] [<ffffffff810c0d82>] do_last+0x220/0x573 Jun 26 22:27:30 [368353.530119] [<ffffffff810c17c3>] path_openat+0xcb/0x319 Jun 26 22:27:30 [368353.530119] [<ffffffff810c1aff>] do_filp_open+0x38/0x84 Jun 26 22:27:30 [368353.530119] [<ffffffff810cb079>] ? alloc_fd+0x6f/0xfe Jun 26 22:27:30 [368353.530119] [<ffffffff810b54bf>] do_sys_open+0x70/0x102 Jun 26 22:27:30 [368353.530119] [<ffffffff810b556d>] sys_open+0x1c/0x1e Jun 26 22:27:30 [368353.530119] [<ffffffff8125fa22>] system_call_fastpath+0x16/0x1b -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
