Here is one more log with crash information. All of my disks are
1.5TB, Seagates and WD. The expander did not affect the crashes, i had
the same issues with direct attached drives.
On 6/6/2010 3:13 μμ, Jelle de Jong wrote:
Dear Srini,
I spent a few weeks catering information and did some intensive
testing the last few days.
Srinivas Naga Venkatasatya Pasagadugula wrote, on 06-05-10 08:01:
1. Is this is the problem with only WD SATA drives? (I don't have WD SATA drives to reproduce this issue.)
2. Whether the problem is with direct attached SATA drives or drives connected in expanders also?
3. Could you please provide me the "dmesg" log or "/var/log/messages" log.
4. How much the capacity of SATA drives connected to controller?
5. Your HBA is having 6440 chipset?
With a few tricks I managed to boot my OS from the mvsas controller. I
got eleven different sata disks attached from a 4-port mini-sas
backplate without expander to the two Marvel 88SE63xx/64xx mvsas
controllers in my system.
I managed to add five mdadm raid1 arrays without added the actual
active sync device (so one disk for each array) I build my lvm systems
on top of this and did a lot of file transfers for testing.
This worked stable enough, there are HDIO_GET_IDENTITY errors during
boot and operation, but the hard disks seem to be working.
So to debug if issues are related to some brand of harddisk, I started
to add WDC, Hitachi, SAMSUNG, Maxtor and Seagate disks to the
respected raid1 array, the disks are of different sizes (320GB, 500GB,
1TB.)
The sync starts and will fail directly or a while later on failures in
the mvsas driver. I attached an failure example as attachment.
The failures are of grave severity, I have lost complete lvm2 volumes
and raid arrays during testing.
Do you also have a SuperMicro AOC-SASLP-MV8 controller for testing?
I would love to use the controllers for production, but they are
currently unstable. I hope this information helps to solve the mvsas
issues?
With kind regards,
Jelle de Jong
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
sd 10:0:1:0: [sdc] Unhandled error code
sd 10:0:1:0: [sdc] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:1:0: [sdc] CDB: cdb[0]=0x28: 28 00 1c b2 86 6f 00 03 00 00
end_request: I/O error, dev sdc, sector 481461871
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 62 03 00 00 04 00 00
end_request: I/O error, dev sdf, sector 442630912
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 61 ff 00 00 04 00 00
end_request: I/O error, dev sdf, sector 442629888
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 61 fb 00 00 04 00 00
end_request: I/O error, dev sdf, sector 442628864
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 61 f7 00 00 04 00 00
end_request: I/O error, dev sdf, sector 442627840
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 61 f3 00 00 04 00 00
end_request: I/O error, dev sdf, sector 442626816
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 61 ef 00 00 04 00 00
end_request: I/O error, dev sdf, sector 442625792
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 00 00 c8 c8 00 00 08 00
end_request: I/O error, dev sdf, sector 51400
metapage_write_end_io: I/O error
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 00 00 db 90 00 00 58 00
end_request: I/O error, dev sdf, sector 56208
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
INFO: task filezilla:18572 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
filezilla D ffffffff810dc2d0 0 18572 2063 0x00000000
ffff8800175b3768 0000000000000082 00000000000530d7 0000000000000008
0000000000800020 ffff88003ca52cb0 ffff8800175b3798 ffffffff811b6fa4
ffff8800175b3740 ffff8800175b3fd8 ffff8800175b2000 ffff8800175b2000
Call Trace:
[<ffffffff811b6fa4>] ? generic_make_request+0x184/0x4f0
[<ffffffff8107d739>] ? ktime_get_ts+0xa9/0xe0
[<ffffffff810dc2d0>] ? sync_page+0x0/0x50
[<ffffffff8135ad5e>] io_schedule+0x6e/0xb0
[<ffffffff810dc30d>] sync_page+0x3d/0x50
[<ffffffff8135b382>] __wait_on_bit_lock+0x52/0xb0
[<ffffffff810dc2b2>] __lock_page+0x62/0x70
[<ffffffff810732b0>] ? wake_bit_function+0x0/0x40
[<ffffffff810dc9a9>] do_read_cache_page+0x159/0x180
[<ffffffffa030d210>] ? metapage_readpage+0x0/0x180 [jfs]
[<ffffffff810dca04>] read_cache_page_async+0x14/0x20
[<ffffffff810dca19>] read_cache_page+0x9/0x20
[<ffffffffa030ddb5>] __get_metapage+0x95/0x5a0 [jfs]
[<ffffffff8135d1b4>] ? __down_read+0xd4/0xd6
[<ffffffffa02f87eb>] ? xtLookup+0x18b/0x1a0 [jfs]
[<ffffffffa0304227>] dbAlloc+0x147/0x480 [jfs]
[<ffffffffa030cb92>] extAlloc+0x162/0x4d0 [jfs]
[<ffffffff8111d740>] ? mem_cgroup_cache_charge+0x140/0x1e0
[<ffffffffa02f4b21>] jfs_get_block+0x1c1/0x220 [jfs]
[<ffffffff8114dd0a>] nobh_write_begin+0x1ea/0x4b0
[<ffffffff8114ae16>] ? __set_page_dirty+0x76/0xd0
[<ffffffffa02f461e>] jfs_write_begin+0x1e/0x20 [jfs]
[<ffffffffa02f4960>] ? jfs_get_block+0x0/0x220 [jfs]
[<ffffffff810db70d>] generic_file_buffered_write+0x10d/0x280
[<ffffffff810581f2>] ? current_fs_time+0x22/0x30
[<ffffffff810dd658>] __generic_file_aio_write+0x238/0x450
[<ffffffff8135baad>] ? __mutex_lock_slowpath+0x26d/0x370
[<ffffffff810dd8d4>] generic_file_aio_write+0x64/0xd0
[<ffffffff81122002>] do_sync_write+0xd2/0x110
[<ffffffff8135dc8e>] ? common_interrupt+0xe/0x13
[<ffffffff8119fcb1>] ? security_file_permission+0x11/0x20
[<ffffffff81122b48>] vfs_write+0xb8/0x1a0
[<ffffffff81122d0c>] sys_write+0x4c/0x80
[<ffffffff81009fc2>] system_call_fastpath+0x16/0x1b
INFO: task filezilla:18574 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
filezilla D ffffffff810dc2d0 0 18574 2063 0x00000000
ffff88003c967768 0000000000000082 0000000000045ca7 0000000000000008
0000000000800020 ffff88003ca52cb0 ffff88003c967798 ffffffff811b6fa4
ffff88003c967740 ffff88003c967fd8 ffff88003c966000 ffff88003c966000
Call Trace:
[<ffffffff811b6fa4>] ? generic_make_request+0x184/0x4f0
[<ffffffff8107d739>] ? ktime_get_ts+0xa9/0xe0
[<ffffffff810dc2d0>] ? sync_page+0x0/0x50
[<ffffffff8135ad5e>] io_schedule+0x6e/0xb0
[<ffffffff810dc30d>] sync_page+0x3d/0x50
[<ffffffff8135b382>] __wait_on_bit_lock+0x52/0xb0
[<ffffffff810dc2b2>] __lock_page+0x62/0x70
[<ffffffff810732b0>] ? wake_bit_function+0x0/0x40
[<ffffffff810dc9a9>] do_read_cache_page+0x159/0x180
[<ffffffffa030d210>] ? metapage_readpage+0x0/0x180 [jfs]
[<ffffffff810dca04>] read_cache_page_async+0x14/0x20
[<ffffffff810dca19>] read_cache_page+0x9/0x20
[<ffffffffa030ddb5>] __get_metapage+0x95/0x5a0 [jfs]
[<ffffffff8135d1b4>] ? __down_read+0xd4/0xd6
[<ffffffffa02f86ed>] ? xtLookup+0x8d/0x1a0 [jfs]
[<ffffffffa0304227>] dbAlloc+0x147/0x480 [jfs]
[<ffffffffa030cb92>] extAlloc+0x162/0x4d0 [jfs]
[<ffffffff8111d740>] ? mem_cgroup_cache_charge+0x140/0x1e0
[<ffffffffa02f4b21>] jfs_get_block+0x1c1/0x220 [jfs]
[<ffffffff8114dd0a>] nobh_write_begin+0x1ea/0x4b0
[<ffffffff8114ae16>] ? __set_page_dirty+0x76/0xd0
[<ffffffffa02f461e>] jfs_write_begin+0x1e/0x20 [jfs]
[<ffffffffa02f4960>] ? jfs_get_block+0x0/0x220 [jfs]
[<ffffffff810db70d>] generic_file_buffered_write+0x10d/0x280
[<ffffffff810581f2>] ? current_fs_time+0x22/0x30
[<ffffffff810dd658>] __generic_file_aio_write+0x238/0x450
[<ffffffff8135baad>] ? __mutex_lock_slowpath+0x26d/0x370
[<ffffffff810dd8d4>] generic_file_aio_write+0x64/0xd0
[<ffffffff81122002>] do_sync_write+0xd2/0x110
[<ffffffff8135dc8e>] ? common_interrupt+0xe/0x13
[<ffffffff8135dc8e>] ? common_interrupt+0xe/0x13
[<ffffffff8119fcb1>] ? security_file_permission+0x11/0x20
[<ffffffff81122b48>] vfs_write+0xb8/0x1a0
[<ffffffff81122d0c>] sys_write+0x4c/0x80
[<ffffffff81009fc2>] system_call_fastpath+0x16/0x1b
INFO: task smbd:18658 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
smbd D ffff88000190f948 0 18658 16782 0x00000000
ffff8800328cd608 0000000000000086 0000000000000000 ffffffff812772b2
ffff8800328cd5a8 ffffffffa000694a ffff880000000000 ffff88003c921830
00000001061958d0 ffff8800328cdfd8 ffff8800328cc000 ffff8800328cc000
Call Trace:
[<ffffffff812772b2>] ? put_device+0x12/0x20
[<ffffffffa000694a>] ? scsi_request_fn+0xaa/0x4d0 [scsi_mod]
[<ffffffff810dc2d0>] ? sync_page+0x0/0x50
[<ffffffff8135ad5e>] io_schedule+0x6e/0xb0
[<ffffffff810dc30d>] sync_page+0x3d/0x50
[<ffffffff8135b382>] __wait_on_bit_lock+0x52/0xb0
[<ffffffff810dc2b2>] __lock_page+0x62/0x70
[<ffffffff810732b0>] ? wake_bit_function+0x0/0x40
[<ffffffff810dc9a9>] do_read_cache_page+0x159/0x180
[<ffffffffa030d210>] ? metapage_readpage+0x0/0x180 [jfs]
[<ffffffff810dca04>] read_cache_page_async+0x14/0x20
[<ffffffff810dca19>] read_cache_page+0x9/0x20
[<ffffffffa030ddb5>] __get_metapage+0x95/0x5a0 [jfs]
[<ffffffffa0309851>] dtSearch+0x781/0xb00 [jfs]
[<ffffffffa02f7503>] jfs_lookup+0x103/0x150 [jfs]
[<ffffffff81137648>] ? d_alloc+0x158/0x1c0
[<ffffffff8112c64d>] __lookup_hash+0xed/0x150
[<ffffffff8112cb95>] lookup_one_len+0x75/0xb0
[<ffffffffa03a114d>] vfsub_lookup_one_len+0x1d/0x50 [aufs]
[<ffffffffa03a7ae2>] au_lkup_one+0x32/0xf0 [aufs]
[<ffffffff810f9619>] ? handle_mm_fault+0x6d9/0xa30
[<ffffffff8135baad>] ? __mutex_lock_slowpath+0x26d/0x370
[<ffffffff8135d0c5>] ? __down_write_nested+0xd5/0xe0
[<ffffffffa03a526f>] au_wh_test+0x1f/0xd0 [aufs]
[<ffffffff8135bbc1>] ? mutex_lock+0x11/0x30
[<ffffffffa03a8286>] au_lkup_dentry+0x336/0x530 [aufs]
[<ffffffffa03af3cf>] aufs_lookup+0xef/0x230 [aufs]
[<ffffffff8112c4cb>] do_lookup+0x1db/0x270
[<ffffffff8112eb59>] link_path_walk+0x679/0xe50
[<ffffffff8112f502>] path_walk+0x62/0xe0
[<ffffffff8112f6c3>] do_path_lookup+0x53/0xa0
[<ffffffff8112f735>] kern_path+0x25/0x50
[<ffffffff811cb30b>] ? cpumask_any_but+0x2b/0x40
[<ffffffff810fdbb0>] ? unmap_region+0x170/0x1a0
[<ffffffff81151629>] lookup_bdev+0x39/0xb0
[<ffffffff8112e26e>] ? getname+0x1ce/0x230
[<ffffffff81174b03>] sys_quotactl+0xb3/0x3d0
[<ffffffff81077df9>] ? up_write+0x9/0x10
[<ffffffff81009fc2>] system_call_fastpath+0x16/0x1b
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
sd 10:0:6:0: [sdh] Unhandled error code
sd 10:0:6:0: [sdh] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:6:0: [sdh] CDB: cdb[0]=0x28: 28 00 27 63 5d 6f 00 00 08 00
end_request: I/O error, dev sdh, sector 660823407
metapage_read_end_io: I/O error
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
INFO: task filezilla:18572 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
filezilla D ffffffff810dc2d0 0 18572 2063 0x00000000
ffff8800175b3768 0000000000000082 00000000000530d7 0000000000000008
0000000000800020 ffff88003ca52cb0 ffff8800175b3798 ffffffff811b6fa4
ffff8800175b3740 ffff8800175b3fd8 ffff8800175b2000 ffff8800175b2000
Call Trace:
[<ffffffff811b6fa4>] ? generic_make_request+0x184/0x4f0
[<ffffffff8107d739>] ? ktime_get_ts+0xa9/0xe0
[<ffffffff810dc2d0>] ? sync_page+0x0/0x50
[<ffffffff8135ad5e>] io_schedule+0x6e/0xb0
[<ffffffff810dc30d>] sync_page+0x3d/0x50
[<ffffffff8135b382>] __wait_on_bit_lock+0x52/0xb0
[<ffffffff810dc2b2>] __lock_page+0x62/0x70
[<ffffffff810732b0>] ? wake_bit_function+0x0/0x40
[<ffffffff810dc9a9>] do_read_cache_page+0x159/0x180
[<ffffffffa030d210>] ? metapage_readpage+0x0/0x180 [jfs]
[<ffffffff810dca04>] read_cache_page_async+0x14/0x20
[<ffffffff810dca19>] read_cache_page+0x9/0x20
[<ffffffffa030ddb5>] __get_metapage+0x95/0x5a0 [jfs]
[<ffffffff8135d1b4>] ? __down_read+0xd4/0xd6
[<ffffffffa02f87eb>] ? xtLookup+0x18b/0x1a0 [jfs]
[<ffffffffa0304227>] dbAlloc+0x147/0x480 [jfs]
[<ffffffffa030cb92>] extAlloc+0x162/0x4d0 [jfs]
[<ffffffff8111d740>] ? mem_cgroup_cache_charge+0x140/0x1e0
[<ffffffffa02f4b21>] jfs_get_block+0x1c1/0x220 [jfs]
[<ffffffff8114dd0a>] nobh_write_begin+0x1ea/0x4b0
[<ffffffff8114ae16>] ? __set_page_dirty+0x76/0xd0
[<ffffffffa02f461e>] jfs_write_begin+0x1e/0x20 [jfs]
[<ffffffffa02f4960>] ? jfs_get_block+0x0/0x220 [jfs]
[<ffffffff810db70d>] generic_file_buffered_write+0x10d/0x280
[<ffffffff810581f2>] ? current_fs_time+0x22/0x30
[<ffffffff810dd658>] __generic_file_aio_write+0x238/0x450
[<ffffffff8135baad>] ? __mutex_lock_slowpath+0x26d/0x370
[<ffffffff810dd8d4>] generic_file_aio_write+0x64/0xd0
[<ffffffff81122002>] do_sync_write+0xd2/0x110
[<ffffffff8135dc8e>] ? common_interrupt+0xe/0x13
[<ffffffff8119fcb1>] ? security_file_permission+0x11/0x20
[<ffffffff81122b48>] vfs_write+0xb8/0x1a0
[<ffffffff81122d0c>] sys_write+0x4c/0x80
[<ffffffff81009fc2>] system_call_fastpath+0x16/0x1b
INFO: task filezilla:18574 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
filezilla D ffffffff810dc2d0 0 18574 2063 0x00000000
ffff88003c967768 0000000000000082 0000000000045ca7 0000000000000008
0000000000800020 ffff88003ca52cb0 ffff88003c967798 ffffffff811b6fa4
ffff88003c967740 ffff88003c967fd8 ffff88003c966000 ffff88003c966000
Call Trace:
[<ffffffff811b6fa4>] ? generic_make_request+0x184/0x4f0
[<ffffffff8107d739>] ? ktime_get_ts+0xa9/0xe0
[<ffffffff810dc2d0>] ? sync_page+0x0/0x50
[<ffffffff8135ad5e>] io_schedule+0x6e/0xb0
[<ffffffff810dc30d>] sync_page+0x3d/0x50
[<ffffffff8135b382>] __wait_on_bit_lock+0x52/0xb0
[<ffffffff810dc2b2>] __lock_page+0x62/0x70
[<ffffffff810732b0>] ? wake_bit_function+0x0/0x40
[<ffffffff810dc9a9>] do_read_cache_page+0x159/0x180
[<ffffffffa030d210>] ? metapage_readpage+0x0/0x180 [jfs]
[<ffffffff810dca04>] read_cache_page_async+0x14/0x20
[<ffffffff810dca19>] read_cache_page+0x9/0x20
[<ffffffffa030ddb5>] __get_metapage+0x95/0x5a0 [jfs]
[<ffffffff8135d1b4>] ? __down_read+0xd4/0xd6
[<ffffffffa02f86ed>] ? xtLookup+0x8d/0x1a0 [jfs]
[<ffffffffa0304227>] dbAlloc+0x147/0x480 [jfs]
[<ffffffffa030cb92>] extAlloc+0x162/0x4d0 [jfs]
[<ffffffff8111d740>] ? mem_cgroup_cache_charge+0x140/0x1e0
[<ffffffffa02f4b21>] jfs_get_block+0x1c1/0x220 [jfs]
[<ffffffff8114dd0a>] nobh_write_begin+0x1ea/0x4b0
[<ffffffff8114ae16>] ? __set_page_dirty+0x76/0xd0
[<ffffffffa02f461e>] jfs_write_begin+0x1e/0x20 [jfs]
[<ffffffffa02f4960>] ? jfs_get_block+0x0/0x220 [jfs]
[<ffffffff810db70d>] generic_file_buffered_write+0x10d/0x280
[<ffffffff810581f2>] ? current_fs_time+0x22/0x30
[<ffffffff810dd658>] __generic_file_aio_write+0x238/0x450
[<ffffffff8135baad>] ? __mutex_lock_slowpath+0x26d/0x370
[<ffffffff810dd8d4>] generic_file_aio_write+0x64/0xd0
[<ffffffff81122002>] do_sync_write+0xd2/0x110
[<ffffffff8135dc8e>] ? common_interrupt+0xe/0x13
[<ffffffff8135dc8e>] ? common_interrupt+0xe/0x13
[<ffffffff8119fcb1>] ? security_file_permission+0x11/0x20
[<ffffffff81122b48>] vfs_write+0xb8/0x1a0
[<ffffffff81122d0c>] sys_write+0x4c/0x80
[<ffffffff81009fc2>] system_call_fastpath+0x16/0x1b
INFO: task smbd:18659 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
smbd D ffff88000180f948 0 18659 16782 0x00000000
ffff8800328e9b18 0000000000000086 0000000000000000 ffff8800328e9db8
0000000020000000 0000000000000000 0000000000000000 ffff88003c9209f0
000000010619a86e ffff8800328e9fd8 ffff8800328e8000 ffff8800328e8000
Call Trace:
[<ffffffff8135d085>] __down_write_nested+0x95/0xe0
[<ffffffff8135d0db>] __down_write+0xb/0x10
[<ffffffff8135c6d9>] down_write+0x9/0x10
[<ffffffffa03a74c7>] di_write_lock+0x27/0x50 [aufs]
[<ffffffffa03983f6>] aufs_read_lock+0x106/0x110 [aufs]
[<ffffffff81077e09>] ? up_read+0x9/0x10
[<ffffffffa03afdd3>] ? aufs_permission+0x313/0x3d0 [aufs]
[<ffffffffa03a8ae1>] aufs_d_revalidate+0x31/0x450 [aufs]
[<ffffffff8112eab2>] link_path_walk+0x5d2/0xe50
[<ffffffff8112f502>] path_walk+0x62/0xe0
[<ffffffff8112f6c3>] do_path_lookup+0x53/0xa0
[<ffffffff81130302>] user_path_at+0x52/0xa0
[<ffffffff8113db14>] ? mntput_no_expire+0x24/0x100
[<ffffffff811d1276>] ? __up_read+0xa6/0xd0
[<ffffffff81077e09>] ? up_read+0x9/0x10
[<ffffffff81126927>] vfs_fstatat+0x37/0x70
[<ffffffff8113db14>] ? mntput_no_expire+0x24/0x100
[<ffffffff81126a86>] vfs_stat+0x16/0x20
[<ffffffff81126aaf>] sys_newstat+0x1f/0x50
[<ffffffff8112c0ec>] ? path_put+0x2c/0x40
[<ffffffff81120ee1>] ? sys_chdir+0x51/0x80
[<ffffffff81009fc2>] system_call_fastpath+0x16/0x1b
INFO: task smbd:18660 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
smbd D ffff88000190f948 0 18660 16782 0x00000000
ffff88003a767d18 0000000000000086 0000000000000000 ffff88003a767d08
000001c73a767cb8 000001c8000001c8 000000083a767dd8 ffff88003c922d90
000000010619fccf ffff88003a767fd8 ffff88003a766000 ffff88003a766000
Call Trace:
[<ffffffff8135d175>] __down_read+0x95/0xd6
[<ffffffff8135c6e9>] down_read+0x9/0x10
[<ffffffffa03a76b6>] di_read_lock+0x26/0x90 [aufs]
[<ffffffffa03af5f2>] aufs_getattr+0xe2/0x4a0 [aufs]
[<ffffffff8113030d>] ? user_path_at+0x5d/0xa0
[<ffffffff811268bc>] vfs_getattr+0x4c/0x80
[<ffffffff81126948>] vfs_fstatat+0x58/0x70
[<ffffffff811269c9>] vfs_lstat+0x19/0x20
[<ffffffff811269ef>] sys_newlstat+0x1f/0x50
[<ffffffff81009fc2>] system_call_fastpath+0x16/0x1b
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 62 12 a0 00 04 00 00
end_request: I/O error, dev sdf, sector 442634912
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 62 0e a0 00 04 00 00
end_request: I/O error, dev sdf, sector 442633888
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 62 0b 00 00 03 a0 00
end_request: I/O error, dev sdf, sector 442632960
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 62 07 00 00 04 00 00
end_request: I/O error, dev sdf, sector 442631936
sd 10:0:1:0: [sdc] Unhandled error code
sd 10:0:1:0: [sdc] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:1:0: [sdc] CDB: cdb[0]=0x28: 28 00 1c b2 87 6f 00 00 08 00
end_request: I/O error, dev sdc, sector 481462127
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
sd 10:0:1:0: [sdc] Unhandled error code
sd 10:0:1:0: [sdc] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:1:0: [sdc] CDB: cdb[0]=0x28: 28 00 00 04 5c a7 00 00 08 00
end_request: I/O error, dev sdc, sector 285863
metapage_read_end_io: I/O error
sd 10:0:1:0: [sdc] Unhandled error code
sd 10:0:1:0: [sdc] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:1:0: [sdc] CDB: cdb[0]=0x28: 28 00 00 05 30 d7 00 00 08 00
end_request: I/O error, dev sdc, sector 340183
metapage_read_end_io: I/O error
sd 10:0:1:0: [sdc] Unhandled error code
sd 10:0:1:0: [sdc] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:1:0: [sdc] CDB: cdb[0]=0x2a: 2a 00 ae a6 3d 67 00 00 08 00
end_request: I/O error, dev sdc, sector 2930130279
