Hi all,
There is a issue which may be related to CONFIG_SCSI_MQ_DEFAULT: before
we developed DIF/DIX feature on kernel 4.18 (disable
CONFIG_SCSI_MQ_DEFAULT default), and
it works well. But when we switch to kernel 4.19-rc1 and 4.20-rc1, Call
trace as follow occurs when running fio and if disable config
CONFIG_SCSI_MQ_DEFAULT,
then it works well. Also if switch ioengine=libaio to ioengine=psync, it
seems also work well. Do you have any idea or encounter similar issue?
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
fio 2.0.5
Starting 12 processes
[ 629.210506] Unable to handle kernel paging request at virtual address
0000ffff8027e048
[ 629.210506] Unable to handle kernel paging request at virtual address
0000ffff8027e048
[ 629.226373] Mem abort info:
[ 629.226373] Mem abort info:
[ 629.231952] ESR = 0x96000006
[ 629.231952] ESR = 0x96000006
[ 629.238052] Exception class = DABT (current EL), IL = 32 bits
[ 629.238052] Exception class = DABT (current EL), IL = 32 bits
[ 629.249898] SET = 0, FnV = 0
[ 629.249898] SET = 0, FnV = 0
[ 629.255998] EA = 0, S1PTW = 0
[ 629.255998] EA = 0, S1PTW = 0
[ 629.262272] Data abort info:
[ 629.262272] Data abort info:
[ 629.268023] ISV = 0, ISS = 0x00000006
[ 629.268023] ISV = 0, ISS = 0x00000006
[ 629.275690] CM = 0, WnR = 0
[ 629.275690] CM = 0, WnR = 0
[ 629.281617] user pgtable: 4k pages, 48-bit VAs, pgdp = 0000000085c91728
[ 629.281617] user pgtable: 4k pages, 48-bit VAs, pgdp = 0000000085c91728
[ 629.294857] [0000ffff8027e048] pgd=00000027a8644003,
pud=00000027a85ea003, pmd=0000000000000000
[ 629.294857] [0000ffff8027e048] pgd=00000027a8644003,
pud=00000027a85ea003, pmd=0000000000000000
[ 629.312278] Internal error: Oops: 96000006 [#1] PREEMPT SMP
[ 629.312278] Internal error: Oops: 96000006 [#1] PREEMPT SMP
[ 629.323427] Modules linked in: hisi_sas_v3_hw [last unloaded:
hisi_sas_v3_hw]
[ 629.323427] Modules linked in: hisi_sas_v3_hw [last unloaded:
hisi_sas_v3_hw]
[ 629.337713] CPU: 13 PID: 4465 Comm: fio Not tainted
4.20.0-rc1-15093-ge876dec #1067
[ 629.337713] CPU: 13 PID: 4465 Comm: fio Not tainted
4.20.0-rc1-15093-ge876dec #1067
[ 629.353040] Hardware name: Huawei D06/D06, BIOS Hisilicon D06 UEFI
RC0 - B601 (V6.01) 11/08/2018
[ 629.353040] Hardware name: Huawei D06/D06, BIOS Hisilicon D06 UEFI
RC0 - B601 (V6.01) 11/08/2018
[ 629.370633] pstate: 80400009 (Nzcv daif +PAN -UAO)
[ 629.370633] pstate: 80400009 (Nzcv daif +PAN -UAO)
[ 629.380218] pc : deadline_remove_request+0x2c/0xd0
[ 629.380218] pc : deadline_remove_request+0x2c/0xd0
[ 629.389799] lr : dd_dispatch_request+0x184/0x1e0
[ 629.389799] lr : dd_dispatch_request+0x184/0x1e0
[ 629.399031] sp : ffff0000275a3aa0
[ 629.399031] sp : ffff0000275a3aa0
[ 629.405651] x29: ffff0000275a3aa0 x28: ffff8027dc230d40
[ 629.405651] x29: ffff0000275a3aa0 x28: ffff8027dc230d40
[ 629.416278] x27: 0000000000000001 x26: ffff8027dc230d40
[ 629.416278] x27: 0000000000000001 x26: ffff8027dc230d40
[ 629.426903] x25: 0000000000000000 x24: ffff0000094f1000
[ 629.426903] x25: 0000000000000000 x24: ffff0000094f1000
[ 629.437529] x23: ffff8027a1e1d140 x22: 0000000000000006
[ 629.437529] x23: ffff8027a1e1d140 x22: 0000000000000006
[ 629.448155] x21: ffff7e009ec69842 x20: ffff8027a1e1d100
[ 629.448155] x21: ffff7e009ec69842 x20: ffff8027a1e1d100
[ 629.458781] x19: ffff8027a1e1d100 x18: 0000000000000000
[ 629.458781] x19: ffff8027a1e1d100 x18: 0000000000000000
[ 629.469406] x17: 000000009e8c9c80 x16: 00000027a3272000
[ 629.469406] x17: 000000009e8c9c80 x16: 00000027a3272000
[ 629.480032] x15: ffff7e009fff9534 x14: ffff7e009ec698c0
[ 629.480032] x15: ffff7e009fff9534 x14: ffff7e009ec698c0
[ 629.490658] x13: ffff7e009ec69880 x12: 0000000000000009
[ 629.490658] x13: ffff7e009ec69880 x12: 0000000000000009
[ 629.501284] x11: 0000000000000000 x10: ffff8027a7c03560
[ 629.501284] x11: 0000000000000000 x10: ffff8027a7c03560
[ 629.511910] x9 : ffff8027a7c007b8 x8 : 0000000000000007
[ 629.511910] x9 : ffff8027a7c007b8 x8 : 0000000000000007
[ 629.522535] x7 : ffff8027a7c011c8 x6 : 0000000000000000
[ 629.522535] x7 : ffff8027a7c011c8 x6 : 0000000000000000
[ 629.533161] x5 : 0000000000000000 x4 : 0000ffff8027e040
[ 629.533161] x5 : 0000000000000000 x4 : 0000ffff8027e040
[ 629.543787] x3 : ffff8027a1e20040 x2 : ffff8027da87f6a0
[ 629.543787] x3 : ffff8027a1e20040 x2 : ffff8027da87f6a0
[ 629.554413] x1 : ffff8027a1e1d140 x0 : ffff8027a1e1d160
[ 629.554413] x1 : ffff8027a1e1d140 x0 : ffff8027a1e1d160
[ 629.565039] Process fio (pid: 4465, stack limit = 0x000000004e5e4c33)
[ 629.565039] Process fio (pid: 4465, stack limit = 0x000000004e5e4c33)
[ 629.577929] Call trace:
[ 629.577929] Call trace:
[ 629.582808] deadline_remove_request+0x2c/0xd0
[ 629.582808] deadline_remove_request+0x2c/0xd0
[ 629.591693] dd_dispatch_request+0x184/0x1e0
[ 629.591693] dd_dispatch_request+0x184/0x1e0
[ 629.600229] blk_mq_do_dispatch_sched+0x4c/0x104
[ 629.600229] blk_mq_do_dispatch_sched+0x4c/0x104
[ 629.609461] blk_mq_sched_dispatch_requests+0x114/0x18c
[ 629.609461] blk_mq_sched_dispatch_requests+0x114/0x18c
[ 629.619913] __blk_mq_run_hw_queue+0xa4/0x11c
[ 629.619913] __blk_mq_run_hw_queue+0xa4/0x11c
[ 629.628624] __blk_mq_delay_run_hw_queue+0x1a4/0x208
[ 629.628624] __blk_mq_delay_run_hw_queue+0x1a4/0x208
[ 629.638553] blk_mq_run_hw_queue+0x9c/0xf8
[ 629.638553] blk_mq_run_hw_queue+0x9c/0xf8
[ 629.646741] blk_mq_sched_insert_requests+0x60/0xac
[ 629.646741] blk_mq_sched_insert_requests+0x60/0xac
[ 629.656495] blk_mq_flush_plug_list+0x144/0x16c
[ 629.656495] blk_mq_flush_plug_list+0x144/0x16c
[ 629.665554] blk_flush_plug_list+0xe8/0x244
[ 629.665554] blk_flush_plug_list+0xe8/0x244
[ 629.673915] blk_finish_plug+0x30/0x40
[ 629.673915] blk_finish_plug+0x30/0x40
[ 629.681408] __arm64_sys_io_submit+0x114/0x218
[ 629.681408] __arm64_sys_io_submit+0x114/0x218
[ 629.690294] el0_svc_common+0x94/0xf0
[ 629.690294] el0_svc_common+0x94/0xf0
[ 629.697611] el0_svc_handler+0x24/0x80
[ 629.697611] el0_svc_handler+0x24/0x80
[ 629.705102] el0_svc+0x8/0x7c0
[ 629.705102] el0_svc+0x8/0x7c0
[ 629.711200] Code: 91010021 f9400ea4 f9402263 f9402662 (f9400496)
[ 629.711200] Code: 91010021 f9400ea4 f9402263 f9402662 (f9400496)
[ 629.723394] ---[ end trace 22423ac28b61ac51 ]---
[ 629.723394] ---[ end trace 22423ac28b61ac51 ]---
[ 629.732631] note: fio[4465] exited with preempt_count 2
[ 629.732631] note: fio[4465] exited with preempt_count 2
[ 629.743112] WARNING: CPU: 9 PID: 4465 at kernel/rcu/tree_plugin.h:337
rcu_note_context_switch+0x38/0x2ec
[ 629.743112] WARNING: CPU: 9 PID: 4465 at kernel/rcu/tree_plugin.h:337
rcu_note_context_switch+0x38/0x2ec
[ 629.762100] Modules linked in: hisi_sas_v3_hw [last unloaded:
hisi_sas_v3_hw]
[ 629.762100] Modules linked in: hisi_sas_v3_hw [last unloaded:
hisi_sas_v3_hw]
[ 629.776387] CPU: 9 PID: 4465 Comm: fio Tainted: G D
4.20.0-rc1-15093-ge876dec #1067
[ 629.776387] CPU: 9 PID: 4465 Comm: fio Tainted: G D
4.20.0-rc1-15093-ge876dec #1067
[ 629.794327] Hardware name: Huawei D06/D06, BIOS Hisilicon D06 UEFI
RC0 - B601 (V6.01) 11/08/2018
[ 629.794327] Hardware name: Huawei D06/D06, BIOS Hisilicon D06 UEFI
RC0 - B601 (V6.01) 11/08/2018
[ 629.811919] pstate: 20400089 (nzCv daIf +PAN -UAO)
[ 629.811919] pstate: 20400089 (nzCv daIf +PAN -UAO)
[ 629.821500] pc : rcu_note_context_switch+0x38/0x2ec
[ 629.821500] pc : rcu_note_context_switch+0x38/0x2ec
[ 629.831258] lr : __schedule+0x88/0x5f8
[ 629.831258] lr : __schedule+0x88/0x5f8
Jo[bs: 12 (f=0)
629.838748] sp : ffff0000275a34d0
[ 629.838748] sp : ffff0000275a34d0
[ 629.846670] x29: ffff0000275a34d0 x28: ffff8027dc230d40
[ 629.846670] x29: ffff0000275a34d0 x28: ffff8027dc230d40
[ 629.857296] x27: 0000000000000000 x26: ffff000008da18e0
[ 629.857296] x27: 0000000000000000 x26: ffff000008da18e0
[ 629.867922] x25: ffff0000094f1000 x24: ffff8027dc230d40
[ 629.867922] x25: ffff0000094f1000 x24: ffff8027dc230d40
[ 629.878548] x23: ffff0000094ca000 x22: ffff0000094d6000
[ 629.878548] x23: ffff0000094ca000 x22: ffff0000094d6000
[ 629.889174] x21: ffff8027e0ab3b00 x20: ffff8027dc230d40
[ 629.889174] x21: ffff8027e0ab3b00 x20: ffff8027dc230d40
[ 629.899800] x19: ffff8027e0ab4700 x18: 0000000000000000
[ 629.899800] x19: ffff8027e0ab4700 x18: 0000000000000000
[ 629.910425] x17: 0000000000000000 x16: 0000000000000000
[ 629.910425] x17: 0000000000000000 x16: 0000000000000000
[ 629.921050] x15: 0000000000000000 x14: 0000000000000000
[ 629.921050] x15: 0000000000000000 x14: 0000000000000000
[ 629.931676] x13: 0000000000000000 x12: 0000000000000000
[ 629.931676] x13: 0000000000000000 x12: 0000000000000000
[ 629.942301] x11: 0000000000000002 x10: 0000000000000970
[ 629.942301] x11: 0000000000000002 x10: 0000000000000970
[ 629.952927] x9 : ffff0000275a3660 x8 : ffff8027dc231710
[ 629.952927] x9 : ffff0000275a3660 x8 : ffff8027dc231710
[ 629.963553] x7 : 000000927fe91094 x6 : ffff8027a0308428
[ 629.963553] x7 : 000000927fe91094 x6 : ffff8027a0308428
[ 629.974178] x5 : 0000000000000000 x4 : 0000000000000001
[ 629.974178] x5 : 0000000000000000 x4 : 0000000000000001
[ 629.984804] x3 : 674238a07d45b500 x2 : 00008027d75dd000
[ 629.984804] x3 : 674238a07d45b500 x2 : 00008027d75dd000
[ 629.995429] x1 : 00008027d75dd000 x0 : 0000000000000001
[ 629.995429] x1 : 00008027d75dd000 x0 : 0000000000000001
[ 630.006055] Call trace:
[ 630.006055] Call trace:
[ 630.010934] rcu_note_context_switch+0x38/0x2ec
[ 630.010934] rcu_note_context_switch+0x38/0x2ec
[ 630.019993] __schedule+0x88/0x5f8
[ 630.019993] __schedule+0x88/0x5f8
[ 630.026788] schedule+0x38/0x9c
[ 630.026788] schedule+0x38/0x9c
[ 630.033060] schedule_timeout+0x1b8/0x320
[ 630.033060] schedule_timeout+0x1b8/0x320
[ 630.041074] wait_for_common+0xb4/0x164
[ 630.041074] wait_for_common+0xb4/0x164
[ 630.048739] wait_for_completion+0x14/0x1c
[ 630.048739] wait_for_completion+0x14/0x1c
[ 630.056928] exit_aio+0x100/0x114
[ 630.056928] exit_aio+0x100/0x114
[ 630.063549] mmput+0x48/0x118
[ 630.063549] mmput+0x48/0x118
[ 630.069473] do_exit+0x280/0x984
[ 630.069473] do_exit+0x280/0x984
[ 630.075920] die+0x1e0/0x20c
[ 630.075920] die+0x1e0/0x20c
[ 630.081670] die_kernel_fault+0x64/0x74
[ 630.081670] die_kernel_fault+0x64/0x74
[ 6Jobs: 12 (f=0)
30.089335] __do_kernel_fault+0x84/0xac
[ 630.089335] __do_kernel_fault+0x84/0xac
[ 630.098477] do_page_fault+0x114/0x3a8
[ 630.098477] do_page_fault+0x114/0x3a8
[ 630.105968] do_translation_fault+0x68/0x74
[ 630.105968] do_translation_fault+0x68/0x74
[ 630.114330] do_mem_abort+0x3c/0x9c
[ 630.114330] do_mem_abort+0x3c/0x9c
[ 630.121298] el1_da+0x20/0x80
[ 630.121298] el1_da+0x20/0x80
[ 630.127223] deadline_remove_request+0x2c/0xd0
[ 630.127223] deadline_remove_request+0x2c/0xd0
[ 630.136108] dd_dispatch_request+0x184/0x1e0
[ 630.136108] dd_dispatch_request+0x184/0x1e0
[ 630.144644] blk_mq_do_dispatch_sched+0x4c/0x104
[ 630.144644] blk_mq_do_dispatch_sched+0x4c/0x104
[ 630.153877] blk_mq_sched_dispatch_requests+0x114/0x18c
[ 630.153877] blk_mq_sched_dispatch_requests+0x114/0x18c
[ 630.164329] __blk_mq_run_hw_queue+0xa4/0x11c
[ 630.164329] __blk_mq_run_hw_queue+0xa4/0x11c
[ 630.173040] __blk_mq_delay_run_hw_queue+0x1a4/0x208
[ 630.173040] __blk_mq_delay_run_hw_queue+0x1a4/0x208
[ 630.182969] blk_mq_run_hw_queue+0x9c/0xf8
[ 630.182969] blk_mq_run_hw_queue+0x9c/0xf8
[ 630.191157] blk_mq_sched_insert_requests+0x60/0xac
[ 630.191157] blk_mq_sched_insert_requests+0x60/0xac
[ 630.200911] blk_mq_flush_plug_list+0x144/0x16c
[ 630.200911] blk_mq_flush_plug_list+0x144/0x16c
[ 630.209970] blk_flush_plug_list+0xe8/0x244
[ 630.209970] blk_flush_plug_list+0xe8/0x244
[ 630.218331] blk_finish_plug+0x30/0x40
[ 630.218331] blk_finish_plug+0x30/0x40
[ 630.225823] __arm64_sys_io_submit+0x114/0x218
[ 630.225823] __arm64_sys_io_submit+0x114/0x218
[ 630.234708] el0_svc_common+0x94/0xf0
[ 630.234708] el0_svc_common+0x94/0xf0
[ 630.242025] el0_svc_handler+0x24/0x80
[ 630.242025] el0_svc_handler+0x24/0x80
[ 630.249516] el0_svc+0x8/0x7c0
[ 630.249516] el0_svc+0x8/0x7c0
[ 630.255613] ---[ end trace 22423ac28b61ac52 ]---
[ 630.255613] ---[ end trace 22423ac28b61ac52 ]---
Creat 4k_rw_depth128_fiotest file sucessfully
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
fio 2.0.5
Starting 12 processes
Jobs: 12 (f=0)
Jobs: 12 (f=0)
Jobs: 12 (f=0)
Jobs: 12 (f=0)
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:08s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:08s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:07s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:07s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:07s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:07s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:06s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:06s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:06s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:06s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:05s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:05s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:05s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:05s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:04s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:04s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:04s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:04s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:03s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:03s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:03s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:03s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:02s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:02s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:02s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:02s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:01s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:01s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:01s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:01s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:00s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:00s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:00s]
Jobs: 12 (f=0): [////////////] [-.-% done] [0K/0K /s] [0 /0 iops] [eta 03m:00s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [5.8% done] [14099K/11669K /s] [3442 /2848 iops] [eta 02m:59s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [6.3% done] [902.5M/828.5M /s] [220K/202K iops] [eta 02m:58s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [6.8% done] [762.6M/982.5M /s] [186K/240K iops] [eta 02m:57s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [7.4% done] [843.9M/722.3M /s] [206K/176K iops] [eta 02m:56s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [7.9% done] [632.4M/631.9M /s] [154K/154K iops] [eta 02m:55s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [8.4% done] [621.2M/620.4M /s] [152K/151K iops] [eta 02m:54s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [8.9% done] [638.9M/636.9M /s] [156K/155K iops] [eta 02m:53s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [9.5% done] [615.1M/615.7M /s] [150K/150K iops] [eta 02m:52s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [10.0% done] [609.8M/606.8M /s] [149K/148K iops] [eta 02m:51s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [10.5% done] [631.8M/636.3M /s] [154K/155K iops] [eta 02m:50s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [11.1% done] [637.5M/633.3M /s] [156K/155K iops] [eta 02m:49s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [11.6% done] [636.7M/635.5M /s] [155K/155K iops] [eta 02m:48s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [12.1% done] [634.1M/636.3M /s] [155K/155K iops] [eta 02m:47s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [12.6% done] [633.2M/636.1M /s] [155K/155K iops] [eta 02m:46s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [13.2% done] [636.2M/633.3M /s] [155K/155K iops] [eta 02m:45s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [13.7% done] [615.3M/614.2M /s] [150K/150K iops] [eta 02m:44s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [14.2% done] [613.8M/611.8M /s] [150K/149K iops] [eta 02m:43s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [14.7% done] [635.8M/634.2M /s] [155K/155K iops] [eta 02m:42s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [15.3% done] [613.9M/615.9M /s] [150K/150K iops] [eta 02m:41s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [15.8% done] [620.9M/618.5M /s] [152K/151K iops] [eta 02m:40s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [16.3% done] [632.9M/635.4M /s] [155K/155K iops] [eta 02m:39s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [16.8% done] [634.6M/634.4M /s] [155K/155K iops] [eta 02m:38s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [17.4% done] [634.4M/632.4M /s] [155K/154K iops] [eta 02m:37s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [17.9% done] [634.9M/636.2M /s] [155K/155K iops] [eta 02m:36s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [18.4% done] [642.1M/644.4M /s] [157K/157K iops] [eta 02m:35s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [18.9% done] [648.2M/647.7M /s] [158K/158K iops] [eta 02m:34s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [19.5% done] [647.4M/647.7M /s] [158K/158K iops] [eta 02m:33s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [20.0% done] [649.6M/647.4M /s] [159K/158K iops] [eta 02m:32s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [20.5% done] [649.4M/648.7M /s] [159K/158K iops] [eta 02m:31s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [21.1% done] [648.8M/649.9M /s] [158K/159K iops] [eta 02m:30s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [21.6% done] [649.4M/648.4M /s] [159K/158K iops] [eta 02m:29s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [22.1% done] [647.1M/648.6M /s] [158K/158K iops] [eta 02m:28s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [22.6% done] [646.5M/650.7M /s] [158K/159K iops] [eta 02m:27s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [23.2% done] [657.5M/658.8M /s] [161K/161K iops] [eta 02m:26s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [23.7% done] [661.1M/657.9M /s] [162K/161K iops] [eta 02m:25s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [24.2% done] [658.9M/663.2M /s] [161K/162K iops] [eta 02m:24s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [24.7% done] [660.2M/658.4M /s] [161K/161K iops] [eta 02m:23s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [25.3% done] [658.3M/661.5M /s] [161K/161K iops] [eta 02m:22s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [25.8% done] [659.5M/661.4M /s] [161K/161K iops] [eta 02m:21s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [26.3% done] [654.9M/655.5M /s] [160K/160K iops] [eta 02m:20s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [26.8% done] [659.6M/656.1M /s] [161K/160K iops] [eta 02m:19s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [27.4% done] [661.8M/660.1M /s] [161K/161K iops] [eta 02m:18s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [27.9% done] [659.1M/661.4M /s] [161K/161K iops] [eta 02m:17s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [28.4% done] [644.3M/643.7M /s] [157K/157K iops] [eta 02m:16s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [28.9% done] [661.2M/659.6M /s] [161K/161K iops] [eta 02m:15s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [29.5% done] [659.3M/658.1M /s] [161K/161K iops] [eta 02m:14s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [30.0% done] [659.2M/659.4M /s] [161K/161K iops] [eta 02m:13s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [30.5% done] [660.6M/660.8M /s] [161K/161K iops] [eta 02m:12s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [31.1% done] [660.3M/660.2M /s] [161K/161K iops] [eta 02m:11s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [31.6% done] [658.8M/661.2M /s] [161K/161K iops] [eta 02m:10s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [32.1% done] [660.6M/658.6M /s] [161K/161K iops] [eta 02m:09s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [32.6% done] [661.5M/658.8M /s] [161K/161K iops] [eta 02m:08s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [33.2% done] [660.1M/659.6M /s] [161K/161K iops] [eta 02m:07s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [33.7% done] [659.4M/659.4M /s] [161K/161K iops] [eta 02m:06s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [34.2% done] [659.2M/659.2M /s] [161K/161K iops] [eta 02m:05s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [34.7% done] [659.9M/659.5M /s] [161K/161K iops] [eta 02m:04s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [35.3% done] [636.6M/638.8M /s] [155K/156K iops] [eta 02m:03s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [35.8% done] [626.7M/627.1M /s] [153K/153K iops] [eta 02m:02s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [36.3% done] [623.1M/627.9M /s] [152K/153K iops] [eta 02m:01s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [36.8% done] [630.3M/625.9M /s] [154K/153K iops] [eta 02m:00s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [37.4% done] [625.6M/629.6M /s] [153K/154K iops] [eta 01m:59s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [37.9% done] [623.7M/626.6M /s] [152K/153K iops] [eta 01m:58s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [38.4% done] [625.5M/624.2M /s] [153K/152K iops] [eta 01m:57s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [38.9% done] [627.2M/623.2M /s] [153K/152K iops] [eta 01m:56s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [39.5% done] [626.6M/627.7M /s] [153K/153K iops] [eta 01m:55s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [40.0% done] [617.9M/616.3M /s] [151K/150K iops] [eta 01m:54s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [40.5% done] [608.9M/609.3M /s] [148K/149K iops] [eta 01m:53s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [41.1% done] [620.5M/618.6M /s] [151K/151K iops] [eta 01m:52s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [41.6% done] [624.9M/628.8M /s] [153K/154K iops] [eta 01m:51s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [42.1% done] [640.2M/643.3M /s] [156K/157K iops] [eta 01m:50s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [42.6% done] [644.8M/648.2M /s] [157K/158K iops] [eta 01m:49s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [43.2% done] [646.7M/647.8M /s] [158K/158K iops] [eta 01m:48s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [43.7% done] [647.2M/644.7M /s] [158K/157K iops] [eta 01m:47s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [44.2% done] [648.7M/646.5M /s] [158K/158K iops] [eta 01m:46s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [44.7% done] [647.4M/647.3M /s] [158K/158K iops] [eta 01m:45s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [45.3% done] [646.6M/648.1M /s] [158K/158K iops] [eta 01m:44s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [45.8% done] [644.8M/647.3M /s] [157K/158K iops] [eta 01m:43s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [46.3% done] [647.8M/649.7M /s] [158K/158K iops] [eta 01m:42s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [46.8% done] [645.2M/647.9M /s] [158K/158K iops] [eta 01m:41s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [47.4% done] [648.1M/646.4M /s] [158K/158K iops] [eta 01m:40s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [47.9% done] [658.9M/656.7M /s] [161K/160K iops] [eta 01m:39s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [48.4% done] [661.6M/658.6M /s] [161K/161K iops] [eta 01m:38s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [48.9% done] [659.3M/663.5M /s] [161K/162K iops] [eta 01m:37s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [49.5% done] [656.1M/663.7M /s] [160K/162K iops] [eta 01m:36s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [50.0% done] [660.9M/657.1M /s] [161K/161K iops] [eta 01m:35s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [50.5% done] [660.2M/662.4M /s] [161K/162K iops] [eta 01m:34s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [51.1% done] [643.4M/643.4M /s] [157K/157K iops] [eta 01m:33s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [51.6% done] [652.4M/653.5M /s] [159K/160K iops] [eta 01m:32s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [52.1% done] [629.3M/630.9M /s] [154K/154K iops] [eta 01m:31s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [52.6% done] [624.3M/625.4M /s] [152K/153K iops] [eta 01m:30s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [53.2% done] [625.3M/622.8M /s] [153K/152K iops] [eta 01m:29s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [53.7% done] [626.3M/624.6M /s] [153K/152K iops] [eta 01m:28s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [54.2% done] [639.3M/639.1M /s] [156K/156K iops] [eta 01m:27s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [54.7% done] [645.5M/648.5M /s] [158K/158K iops] [eta 01m:26s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [55.3% done] [649.6M/652.6M /s] [159K/159K iops] [eta 01m:25s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [55.8% done] [659.1M/661.2M /s] [161K/161K iops] [eta 01m:24s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [56.3% done] [659.8M/659.5M /s] [161K/161K iops] [eta 01m:23s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [56.8% done] [661.5M/660.2M /s] [161K/161K iops] [eta 01m:22s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [57.4% done] [664.5M/657.6M /s] [162K/161K iops] [eta 01m:21s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [57.9% done] [659.2M/661.9M /s] [161K/162K iops] [eta 01m:20s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [58.4% done] [630.4M/632.7M /s] [154K/154K iops] [eta 01m:19s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [58.9% done] [623.3M/626.9M /s] [152K/153K iops] [eta 01m:18s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [59.5% done] [627.7M/626.8M /s] [153K/153K iops] [eta 01m:17s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [60.0% done] [627.9M/629.2M /s] [153K/154K iops] [eta 01m:16s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [60.5% done] [626.7M/625.1M /s] [153K/153K iops] [eta 01m:15s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [61.1% done] [627.9M/624.2M /s] [153K/152K iops] [eta 01m:14s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [61.6% done] [627.1M/626.3M /s] [153K/153K iops] [eta 01m:13s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [62.1% done] [627.2M/629.8M /s] [153K/154K iops] [eta 01m:12s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [62.6% done] [624.8M/625.2M /s] [153K/153K iops] [eta 01m:11s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [63.2% done] [624.9M/624.3M /s] [153K/152K iops] [eta 01m:10s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [63.7% done] [625.6M/626.7M /s] [153K/153K iops] [eta 01m:09s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [64.2% done] [626.7M/626.4M /s] [153K/153K iops] [eta 01m:08s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [64.7% done] [625.1M/630.8M /s] [153K/154K iops] [eta 01m:07s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [65.3% done] [626.3M/626.2M /s] [153K/153K iops] [eta 01m:06s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [65.8% done] [628.7M/627.1M /s] [153K/153K iops] [eta 01m:05s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [66.3% done] [624.5M/625.1M /s] [152K/153K iops] [eta 01m:04s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [66.8% done] [626.6M/626.4M /s] [153K/153K iops] [eta 01m:03s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [67.4% done] [636.6M/636.7M /s] [155K/155K iops] [eta 01m:02s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [67.9% done] [652.4M/652.2M /s] [159K/159K iops] [eta 01m:01s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [68.4% done] [657.3M/659.5M /s] [160K/161K iops] [eta 01m:00s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [68.9% done] [658.5M/658.4M /s] [161K/161K iops] [eta 00m:59s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [69.5% done] [658.8M/657.9M /s] [161K/161K iops] [eta 00m:58s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [70.0% done] [658.1M/658.2M /s] [161K/161K iops] [eta 00m:57s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [70.5% done] [657.1M/659.8M /s] [161K/161K iops] [eta 00m:56s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [71.1% done] [658.2M/657.1M /s] [161K/161K iops] [eta 00m:55s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [71.6% done] [658.9M/659.8M /s] [161K/161K iops] [eta 00m:54s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [72.1% done] [659.6M/661.1M /s] [161K/162K iops] [eta 00m:53s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [72.6% done] [658.8M/662.8M /s] [161K/162K iops] [eta 00m:52s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [73.2% done] [658.7M/660.6M /s] [161K/161K iops] [eta 00m:51s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [73.7% done] [660.7M/660.9M /s] [161K/161K iops] [eta 00m:50s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [74.2% done] [658.9M/660.1M /s] [161K/161K iops] [eta 00m:49s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [74.7% done] [661.2M/659.2M /s] [161K/161K iops] [eta 00m:48s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [75.3% done] [661.5M/659.5M /s] [161K/161K iops] [eta 00m:47s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [75.8% done] [664.1M/659.9M /s] [162K/161K iops] [eta 00m:46s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [76.3% done] [675.6M/674.6M /s] [165K/165K iops] [eta 00m:45s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [76.8% done] [675.2M/674.5M /s] [165K/165K iops] [eta 00m:44s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [77.4% done] [674.7M/676.5M /s] [165K/165K iops] [eta 00m:43s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [77.9% done] [676.4M/673.9M /s] [165K/165K iops] [eta 00m:42s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [78.4% done] [675.6M/674.6M /s] [165K/165K iops] [eta 00m:41s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [78.9% done] [673.4M/674.5M /s] [164K/165K iops] [eta 00m:40s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [79.5% done] [663.8M/662.7M /s] [162K/162K iops] [eta 00m:39s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [80.0% done] [675.5M/674.5M /s] [165K/165K iops] [eta 00m:38s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [80.5% done] [673.1M/674.7M /s] [164K/165K iops] [eta 00m:37s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [81.1% done] [672.6M/672.5M /s] [164K/164K iops] [eta 00m:36s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [81.7% done] [674.8M/673.8M /s] [165K/164K iops] [eta 00m:35s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [82.2% done] [674.3M/673.4M /s] [165K/164K iops] [eta 00m:34s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [82.7% done] [671.5M/674.2M /s] [164K/165K iops] [eta 00m:33s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [83.2% done] [674.3M/673.3M /s] [165K/164K iops] [eta 00m:32s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [83.8% done] [675.4M/673.1M /s] [165K/165K iops] [eta 00m:31s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [84.3% done] [676.4M/673.5M /s] [165K/164K iops] [eta 00m:30s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [84.8% done] [676.4M/675.5M /s] [165K/165K iops] [eta 00m:29s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [85.3% done] [676.3M/675.7M /s] [165K/165K iops] [eta 00m:28s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [85.9% done] [676.8M/676.4M /s] [165K/165K iops] [eta 00m:27s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [86.4% done] [677.6M/672.1M /s] [165K/164K iops] [eta 00m:26s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [86.9% done] [675.1M/675.1M /s] [165K/165K iops] [eta 00m:25s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [87.4% done] [676.4M/675.4M /s] [165K/165K iops] [eta 00m:24s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [88.0% done] [677.7M/677.2M /s] [165K/165K iops] [eta 00m:23s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [88.5% done] [677.6M/676.4M /s] [165K/165K iops] [eta 00m:22s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [89.0% done] [676.8M/676.8M /s] [165K/165K iops] [eta 00m:21s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [89.5% done] [672.8M/680.7M /s] [164K/166K iops] [eta 00m:20s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [90.1% done] [676.5M/677.3M /s] [165K/165K iops] [eta 00m:19s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [90.6% done] [678.1M/676.9M /s] [166K/165K iops] [eta 00m:18s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [91.1% done] [675.8M/677.8M /s] [165K/165K iops] [eta 00m:17s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [91.6% done] [674.4M/675.4M /s] [165K/165K iops] [eta 00m:16s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [92.1% done] [664.6M/663.8M /s] [162K/162K iops] [eta 00m:15s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [92.7% done] [675.5M/671.9M /s] [165K/164K iops] [eta 00m:14s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [93.2% done] [672.7M/672.5M /s] [164K/164K iops] [eta 00m:13s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [93.7% done] [671.4M/668.8M /s] [164K/163K iops] [eta 00m:12s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [94.2% done] [670.4M/672.3M /s] [164K/164K iops] [eta 00m:11s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [94.8% done] [672.8M/675.5M /s] [164K/165K iops] [eta 00m:10s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [95.3% done] [676.2M/670.5M /s] [165K/164K iops] [eta 00m:09s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [95.8% done] [673.9M/675.6M /s] [165K/165K iops] [eta 00m:08s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [96.3% done] [676.3M/672.7M /s] [165K/164K iops] [eta 00m:07s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [96.9% done] [676.8M/671.1M /s] [165K/164K iops] [eta 00m:06s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [97.4% done] [672.5M/673.8M /s] [164K/164K iops] [eta 00m:05s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [97.9% done] [671.4M/669.7M /s] [164K/163K iops] [eta 00m:04s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [98.4% done] [670.4M/674.4M /s] [164K/165K iops] [eta 00m:03s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [99.0% done] [673.7M/674.2M /s] [164K/165K iops] [eta 00m:02s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [99.5% done] [673.2M/673.2M /s] [164K/164K iops] [eta 00m:01s]
Jobs: 12 (f=12): [MMMMMMMMMMMM] [100.0% done] [674.7M/672.4M /s] [165K/164K iops] [eta 00m:00s]
job1: (groupid=0, jobs=12): err= 0: pid=3874
read : io=112337MB, bw=639052KB/s, iops=159758 , runt=180005msec
slat (usec): min=1 , max=4406 , avg= 4.34, stdev= 7.19
clat (usec): min=20 , max=16839 , avg=4888.83, stdev=1496.23
lat (usec): min=28 , max=16841 , avg=4893.26, stdev=1494.28
clat percentiles (usec):
| 1.00th=[ 2928], 5.00th=[ 3248], 10.00th=[ 3312], 20.00th=[ 3408],
| 30.00th=[ 3472], 40.00th=[ 3568], 50.00th=[ 3920], 60.00th=[ 6112],
| 70.00th=[ 6304], 80.00th=[ 6432], 90.00th=[ 6624], 95.00th=[ 6688],
| 99.00th=[ 6944], 99.50th=[ 7136], 99.90th=[ 8256]
bw (KB/s) : min= 0, max=168600, per=8.37%, avg=53494.99, stdev=7987.05
write: io=112343MB, bw=639089KB/s, iops=159767 , runt=180005msec
slat (usec): min=1 , max=3341 , avg= 4.52, stdev= 7.08
clat (usec): min=26 , max=12875 , avg=4714.99, stdev=1449.90
lat (usec): min=34 , max=12877 , avg=4719.60, stdev=1447.95
clat percentiles (usec):
| 1.00th=[ 2896], 5.00th=[ 3152], 10.00th=[ 3216], 20.00th=[ 3280],
| 30.00th=[ 3344], 40.00th=[ 3440], 50.00th=[ 3696], 60.00th=[ 5920],
| 70.00th=[ 6048], 80.00th=[ 6176], 90.00th=[ 6368], 95.00th=[ 6496],
| 99.00th=[ 6688], 99.50th=[ 6880], 99.90th=[ 7968]
bw (KB/s) : min= 0, max=155768, per=8.36%, avg=53397.35, stdev=7730.15
lat (usec) : 50=0.01%, 100=0.01%, 250=0.01%, 500=0.01%, 750=0.03%
lat (usec) : 1000=0.03%
lat (msec) : 2=0.25%, 4=49.89%, 10=49.78%, 20=0.01%
cpu : usr=3.56%, sys=12.37%, ctx=867899, majf=0, minf=3732
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=105.4%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.1%
issued : total=r=28757389/w=28759031/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=112337MB, aggrb=639052KB/s, minb=654389KB/s, maxb=654389KB/s, mint=180005msec, maxt=180005msec
WRITE: io=112343MB, aggrb=639088KB/s, minb=654426KB/s, maxb=654426KB/s, mint=180005msec, maxt=180005msec
Disk stats (read/write):
sdb: ios=1882423/1870026, merge=655365/638486, ticks=7598597/7455054, in_queue=11516156, util=100.00%
sdc: ios=1875581/1859758, merge=666514/652191, ticks=7771243/7603780, in_queue=11852784, util=100.00%
sda: ios=1861462/1870446, merge=648061/669440, ticks=7741259/7625233, in_queue=11829088, util=100.00%
sde: ios=1859610/1871711, merge=648268/669318, ticks=7731183/7621478, in_queue=11840624, util=100.00%
sdd: ios=1875063/1861567, merge=668113/651005, ticks=7768061/7604671, in_queue=11861292, util=100.00%
sdf: ios=1875610/1860054, merge=669361/648896, ticks=7766338/7597224, in_queue=11844000, util=100.00%
sdh: ios=1860733/1872170, merge=648073/669182, ticks=7738473/7627724, in_queue=11853960, util=100.00%
sdi: ios=1866027/1878153, merge=638707/658066, ticks=7562550/7470393, in_queue=11523588, util=100.00%
sdj: ios=1865874/1878173, merge=640898/662800, ticks=7607518/7511112, in_queue=11621248, util=100.00%
sdg: ios=1859649/1872425, merge=646653/664821, ticks=7735257/7641005, in_queue=11860448, util=100.00%
sdl: ios=1875534/1861691, merge=668426/649576, ticks=7761596/7601379, in_queue=11831976, util=100.00%
sdk: ios=1879027/1863265, merge=666816/651064, ticks=7771003/7608856, in_queue=11837400, util=100.00%
root@(none)$ rmmod hisi_sas_v3_hw.ko
[ 601.784539] hisi_sas_v3_hw 0000:74:02.0: dev[14:1] is gone
[ 601.784539] hisi_sas_v3_hw 0000:74:02.0: dev[14:1] is gone
[ 601.795921] sd 3:0:11:0: [sdl] Synchronizing SCSI cache
[ 601.795921] sd 3:0:11:0: [sdl] Synchronizing SCSI cache
[ 601.852572] hisi_sas_v3_hw 0000:74:02.0: dev[13:1] is gone
[ 601.852572] hisi_sas_v3_hw 0000:74:02.0: dev[13:1] is gone
[ 601.863924] sd 3:0:10:0: [sdk] Synchronizing SCSI cache
[ 601.863924] sd 3:0:10:0: [sdk] Synchronizing SCSI cache
[ 601.920588] hisi_sas_v3_hw 0000:74:02.0: dev[12:1] is gone
[ 601.920588] hisi_sas_v3_hw 0000:74:02.0: dev[12:1] is gone
[ 601.931985] sd 3:0:9:0: [sdj] Synchronizing SCSI cache
[ 601.931985] sd 3:0:9:0: [sdj] Synchronizing SCSI cache
[ 602.004558] hisi_sas_v3_hw 0000:74:02.0: dev[11:1] is gone
[ 602.004558] hisi_sas_v3_hw 0000:74:02.0: dev[11:1] is gone
[ 602.015951] sd 3:0:8:0: [sdi] Synchronizing SCSI cache
[ 602.015951] sd 3:0:8:0: [sdi] Synchronizing SCSI cache
[ 602.076536] hisi_sas_v3_hw 0000:74:02.0: dev[10:1] is gone
[ 602.076536] hisi_sas_v3_hw 0000:74:02.0: dev[10:1] is gone
[ 602.087930] sd 3:0:7:0: [sdh] Synchronizing SCSI cache
[ 602.087930] sd 3:0:7:0: [sdh] Synchronizing SCSI cache
[ 602.132543] hisi_sas_v3_hw 0000:74:02.0: dev[9:1] is gone
[ 602.132543] hisi_sas_v3_hw 0000:74:02.0: dev[9:1] is gone
[ 602.143723] sd 3:0:6:0: [sdg] Synchronizing SCSI cache
[ 602.143723] sd 3:0:6:0: [sdg] Synchronizing SCSI cache
[ 602.200555] hisi_sas_v3_hw 0000:74:02.0: dev[8:1] is gone
[ 602.200555] hisi_sas_v3_hw 0000:74:02.0: dev[8:1] is gone
[ 602.211742] sd 3:0:5:0: [sdf] Synchronizing SCSI cache
[ 602.211742] sd 3:0:5:0: [sdf] Synchronizing SCSI cache
[ 602.252523] hisi_sas_v3_hw 0000:74:02.0: dev[7:1] is gone
[ 602.252523] hisi_sas_v3_hw 0000:74:02.0: dev[7:1] is gone
[ 602.263700] sd 3:0:4:0: [sde] Synchronizing SCSI cache
[ 602.263700] sd 3:0:4:0: [sde] Synchronizing SCSI cache
[ 602.312494] hisi_sas_v3_hw 0000:74:02.0: dev[6:1] is gone
[ 602.312494] hisi_sas_v3_hw 0000:74:02.0: dev[6:1] is gone
[ 602.323660] sd 3:0:3:0: [sdd] Synchronizing SCSI cache
[ 602.323660] sd 3:0:3:0: [sdd] Synchronizing SCSI cache
[ 602.376485] hisi_sas_v3_hw 0000:74:02.0: dev[5:1] is gone
[ 602.376485] hisi_sas_v3_hw 0000:74:02.0: dev[5:1] is gone
[ 602.387642] sd 3:0:2:0: [sdc] Synchronizing SCSI cache
[ 602.387642] sd 3:0:2:0: [sdc] Synchronizing SCSI cache
[ 602.440474] hisi_sas_v3_hw 0000:74:02.0: dev[4:1] is gone
[ 602.440474] hisi_sas_v3_hw 0000:74:02.0: dev[4:1] is gone
[ 602.451671] sd 3:0:1:0: [sdb] Synchronizing SCSI cache
[ 602.451671] sd 3:0:1:0: [sdb] Synchronizing SCSI cache
[ 602.512430] hisi_sas_v3_hw 0000:74:02.0: dev[3:1] is gone
[ 602.512430] hisi_sas_v3_hw 0000:74:02.0: dev[3:1] is gone
[ 602.523649] sd 3:0:0:0: [sda] Synchronizing SCSI cache
[ 602.523649] sd 3:0:0:0: [sda] Synchronizing SCSI cache
[ 602.580517] hisi_sas_v3_hw 0000:74:02.0: dev[2:1] is gone
[ 602.580517] hisi_sas_v3_hw 0000:74:02.0: dev[2:1] is gone
[ 602.592684] hisi_sas_v3_hw 0000:74:02.0: dev[1:2] is gone
[ 602.592684] hisi_sas_v3_hw 0000:74:02.0: dev[1:2] is gone
root@(none)$
root@(none)$
root@(none)$
root@(none)$ insmod hisi_sas_v3_hw.ko enable_dif_dix=1
[ 614.945604] scsi host3: hisi_sas_v3_hw
[ 614.945604] scsi host3: hisi_sas_v3_hw
[ 616.159646] hisi_sas_v3_hw 0000:74:02.0: Registering for DIF/DIX type 1/2/3 protection.
[ 616.159646] hisi_sas_v3_hw 0000:74:02.0: Registering for DIF/DIX type 1/2/3 protection.
[ 616.233223] hisi_sas_v3_hw 0000:74:02.0: phyup: phy0 link_rate=11
[ 616.233223] hisi_sas_v3_hw 0000:74:02.0: phyup: phy0 link_rate=11
[ 616.245423] hisi_sas_v3_hw 0000:74:02.0: phyup: phy1 link_rate=11
[ 616.245423] hisi_sas_v3_hw 0000:74:02.0: phyup: phy1 link_rate=11
[ 616.257620] hisi_sas_v3_hw 0000:74:02.0: phyup: phy2 link_rate=11
[ 616.257620] hisi_sas_v3_hw 0000:74:02.0: phyup: phy2 link_rate=11
[ 616.269816] hisi_sas_v3_hw 0000:74:02.0: phyup: phy4 link_rate=11
[ 616.269816] hisi_sas_v3_hw 0000:74:02.0: phyup: phy4 link_rate=11
[ 616.282141] hisi_sas_v3_hw 0000:74:02.0: dev[1:2] found
[ 616.282141] hisi_sas_v3_hw 0000:74:02.0: dev[1:2] found
[ 616.296076] hisi_sas_v3_hw 0000:74:02.0: dev[2:1] found
[ 616.296076] hisi_sas_v3_hw 0000:74:02.0: dev[2:1] found
[ 616.306620] hisi_sas_v3_hw 0000:74:02.0: dev[3:1] found
[ 616.306620] hisi_sas_v3_hw 0000:74:02.0: dev[3:1] found
[ 616.317160] hisi_sas_v3_hw 0000:74:02.0: dev[4:1] found
[ 616.317160] hisi_sas_v3_hw 0000:74:02.0: dev[4:1] found
[ 616.327695] hisi_sas_v3_hw 0000:74:02.0: dev[5:1] found
[ 616.327695] hisi_sas_v3_hw 0000:74:02.0: dev[5:1] found
[ 616.338227] hisi_sas_v3_hw 0000:74:02.0: dev[6:1] found
[ 616.338227] hisi_sas_v3_hw 0000:74:02.0: dev[6:1] found
[ 616.348762] hisi_sas_v3_hw 0000:74:02.0: dev[7:1] found
[ 616.348762] hisi_sas_v3_hw 0000:74:02.0: dev[7:1] found
[ 616.359295] hisi_sas_v3_hw 0000:74:02.0: dev[8:1] found
[ 616.359295] hisi_sas_v3_hw 0000:74:02.0: dev[8:1] found
[ 616.369829] hisi_sas_v3_hw 0000:74:02.0: dev[9:1] found
[ 616.369829] hisi_sas_v3_hw 0000:74:02.0: dev[9:1] found
[ 616.380364] hisi_sas_v3_hw 0000:74:02.0: dev[10:1] found
[ 616.380364] hisi_sas_v3_hw 0000:74:02.0: dev[10:1] found
[ 616.391071] hisi_sas_v3_hw 0000:74:02.0: dev[11:1] found
[ 616.391071] hisi_sas_v3_hw 0000:74:02.0: dev[11:1] found
[ 616.401779] hisi_sas_v3_hw 0000:74:02.0: dev[12:1] found
[ 616.401779] hisi_sas_v3_hw 0000:74:02.0: dev[12:1] found
[ 616.412489] hisi_sas_v3_hw 0000:74:02.0: dev[13:1] found
[ 616.412489] hisi_sas_v3_hw 0000:74:02.0: dev[13:1] found
[ 616.423219] hisi_sas_v3_hw 0000:74:02.0: dev[14:1] found
[ 616.423219] hisi_sas_v3_hw 0000:74:02.0: dev[14:1] found
[ 616.434412] scsi 3:0:0:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 616.434412] scsi 3:0:0:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 616.450976] sd 3:0:0:0: Attached scsi generic sg0 type 0
[ 616.450976] sd 3:0:0:0: Attached scsi generic sg0 type 0
[ 616.451286] sd 3:0:0:0: [sda] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 616.451286] sd 3:0:0:0: [sda] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 616.462192] scsi 3:0:1:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 616.462192] scsi 3:0:1:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 616.476597] sd 3:0:0:0: [sda] 4096-byte physical blocks
[ 616.476597] sd 3:0:0:0: [sda] 4096-byte physical blocks
[ 616.493127] sd 3:0:1:0: Attached scsi generic sg1 type 0
[ 616.493127] sd 3:0:1:0: Attached scsi generic sg1 type 0
[ 616.493450] sd 3:0:1:0: [sdb] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 616.493450] sd 3:0:1:0: [sdb] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 616.493452] sd 3:0:1:0: [sdb] 4096-byte physical blocks
[ 616.493452] sd 3:0:1:0: [sdb] 4096-byte physical blocks
[ 616.493551] sd 3:0:1:0: [sdb] Write Protect is off
[ 616.493551] sd 3:0:1:0: [sdb] Write Protect is off
[ 616.493680] sd 3:0:1:0: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 616.493680] sd 3:0:1:0: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 616.500387] sd 3:0:1:0: [sdb] Attached SCSI disk
[ 616.500387] sd 3:0:1:0: [sdb] Attached SCSI disk
[ 616.503354] sd 3:0:0:0: [sda] Write Protect is off
[ 616.503354] sd 3:0:0:0: [sda] Write Protect is off
[ 616.514415] scsi 3:0:2:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 616.514415] scsi 3:0:2:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 616.528992] sd 3:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 616.528992] sd 3:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 616.539672] sd 3:0:2:0: Attached scsi generic sg2 type 0
[ 616.539672] sd 3:0:2:0: Attached scsi generic sg2 type 0
[ 616.539953] sd 3:0:2:0: [sdc] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 616.539953] sd 3:0:2:0: [sdc] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 616.539955] sd 3:0:2:0: [sdc] 4096-byte physical blocks
[ 616.539955] sd 3:0:2:0: [sdc] 4096-byte physical blocks
[ 616.540052] sd 3:0:2:0: [sdc] Write Protect is off
[ 616.540052] sd 3:0:2:0: [sdc] Write Protect is off
[ 616.540176] sd 3:0:2:0: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 616.540176] sd 3:0:2:0: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 616.546808] sd 3:0:2:0: [sdc] Attached SCSI disk
[ 616.546808] sd 3:0:2:0: [sdc] Attached SCSI disk
[ 616.555549] sd 3:0:0:0: [sda] Attached SCSI disk
[ 616.555549] sd 3:0:0:0: [sda] Attached SCSI disk
[ 616.566511] scsi 3:0:3:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 616.566511] scsi 3:0:3:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 616.715976] sd 3:0:3:0: Attached scsi generic sg3 type 0
[ 616.715976] sd 3:0:3:0: Attached scsi generic sg3 type 0
[ 616.716279] sd 3:0:3:0: [sdd] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 616.716279] sd 3:0:3:0: [sdd] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 616.727163] scsi 3:0:4:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 616.727163] scsi 3:0:4:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 616.741609] sd 3:0:3:0: [sdd] 4096-byte physical blocks
[ 616.741609] sd 3:0:3:0: [sdd] 4096-byte physical blocks
[ 616.758131] sd 3:0:4:0: Attached scsi generic sg4 type 0
[ 616.758131] sd 3:0:4:0: Attached scsi generic sg4 type 0
[ 616.758492] sd 3:0:4:0: [sde] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 616.758492] sd 3:0:4:0: [sde] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 616.758496] sd 3:0:4:0: [sde] 4096-byte physical blocks
[ 616.758496] sd 3:0:4:0: [sde] 4096-byte physical blocks
[ 616.758597] sd 3:0:4:0: [sde] Write Protect is off
[ 616.758597] sd 3:0:4:0: [sde] Write Protect is off
[ 616.758727] sd 3:0:4:0: [sde] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 616.758727] sd 3:0:4:0: [sde] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 616.766077] sd 3:0:4:0: [sde] Attached SCSI disk
[ 616.766077] sd 3:0:4:0: [sde] Attached SCSI disk
[ 616.768431] sd 3:0:3:0: [sdd] Write Protect is off
[ 616.768431] sd 3:0:3:0: [sdd] Write Protect is off
[ 616.779362] scsi 3:0:5:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 616.779362] scsi 3:0:5:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 616.804667] sd 3:0:5:0: Attached scsi generic sg5 type 0
[ 616.804667] sd 3:0:5:0: Attached scsi generic sg5 type 0
[ 616.804700] sd 3:0:5:0: [sdf] Enabling DIF Type 1 protection
[ 616.804700] sd 3:0:5:0: [sdf] Enabling DIF Type 1 protection
[ 616.804977] sd 3:0:5:0: [sdf] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 616.804977] sd 3:0:5:0: [sdf] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 616.804979] sd 3:0:5:0: [sdf] 4096-byte physical blocks
[ 616.804979] sd 3:0:5:0: [sdf] 4096-byte physical blocks
[ 616.805078] sd 3:0:5:0: [sdf] Write Protect is off
[ 616.805078] sd 3:0:5:0: [sdf] Write Protect is off
[ 616.805206] sd 3:0:5:0: [sdf] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 616.805206] sd 3:0:5:0: [sdf] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 616.809934] sd 3:0:5:0: [sdf] Enabling DIX T10-DIF-TYPE1-CRC protection
[ 616.809934] sd 3:0:5:0: [sdf] Enabling DIX T10-DIF-TYPE1-CRC protection
[ 616.811961] sd 3:0:5:0: [sdf] Attached SCSI disk
[ 616.811961] sd 3:0:5:0: [sdf] Attached SCSI disk
[ 616.814040] sd 3:0:3:0: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 616.814040] sd 3:0:3:0: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 616.831533] scsi 3:0:6:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 616.831533] scsi 3:0:6:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 616.847197] sd 3:0:3:0: [sdd] Attached SCSI disk
[ 616.847197] sd 3:0:3:0: [sdd] Attached SCSI disk
[ 616.850133] sd 3:0:6:0: Attached scsi generic sg6 type 0
[ 616.850133] sd 3:0:6:0: Attached scsi generic sg6 type 0
[ 616.850174] sd 3:0:6:0: [sdg] Enabling DIF Type 1 protection
[ 616.850174] sd 3:0:6:0: [sdg] Enabling DIF Type 1 protection
[ 616.850451] sd 3:0:6:0: [sdg] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 616.850451] sd 3:0:6:0: [sdg] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 616.850453] sd 3:0:6:0: [sdg] 4096-byte physical blocks
[ 616.850453] sd 3:0:6:0: [sdg] 4096-byte physical blocks
[ 616.850552] sd 3:0:6:0: [sdg] Write Protect is off
[ 616.850552] sd 3:0:6:0: [sdg] Write Protect is off
[ 616.850681] sd 3:0:6:0: [sdg] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 616.850681] sd 3:0:6:0: [sdg] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 616.855312] sd 3:0:6:0: [sdg] Enabling DIX T10-DIF-TYPE1-CRC protection
[ 616.855312] sd 3:0:6:0: [sdg] Enabling DIX T10-DIF-TYPE1-CRC protection
[ 616.857340] sd 3:0:6:0: [sdg] Attached SCSI disk
[ 616.857340] sd 3:0:6:0: [sdg] Attached SCSI disk
[ 617.102088] scsi 3:0:7:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 617.102088] scsi 3:0:7:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 617.118639] sd 3:0:7:0: Attached scsi generic sg7 type 0
[ 617.118639] sd 3:0:7:0: Attached scsi generic sg7 type 0
[ 617.118951] sd 3:0:7:0: [sdh] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 617.118951] sd 3:0:7:0: [sdh] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 617.129747] scsi 3:0:8:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 617.129747] scsi 3:0:8:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 617.144257] sd 3:0:7:0: [sdh] 4096-byte physical blocks
[ 617.144257] sd 3:0:7:0: [sdh] 4096-byte physical blocks
[ 617.144364] sd 3:0:7:0: [sdh] Write Protect is off
[ 617.144364] sd 3:0:7:0: [sdh] Write Protect is off
[ 617.160903] sd 3:0:8:0: Attached scsi generic sg8 type 0
[ 617.160903] sd 3:0:8:0: Attached scsi generic sg8 type 0
[ 617.160944] sd 3:0:8:0: [sdi] Enabling DIF Type 1 protection
[ 617.160944] sd 3:0:8:0: [sdi] Enabling DIF Type 1 protection
[ 617.161227] sd 3:0:8:0: [sdi] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 617.161227] sd 3:0:8:0: [sdi] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 617.161229] sd 3:0:8:0: [sdi] 4096-byte physical blocks
[ 617.161229] sd 3:0:8:0: [sdi] 4096-byte physical blocks
[ 617.161329] sd 3:0:8:0: [sdi] Write Protect is off
[ 617.161329] sd 3:0:8:0: [sdi] Write Protect is off
[ 617.161458] sd 3:0:8:0: [sdi] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 617.161458] sd 3:0:8:0: [sdi] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 617.166257] sd 3:0:8:0: [sdi] Enabling DIX T10-DIF-TYPE1-CRC protection
[ 617.166257] sd 3:0:8:0: [sdi] Enabling DIX T10-DIF-TYPE1-CRC protection
[ 617.168273] sd 3:0:8:0: [sdi] Attached SCSI disk
[ 617.168273] sd 3:0:8:0: [sdi] Attached SCSI disk
[ 617.180972] scsi 3:0:9:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 617.180972] scsi 3:0:9:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 617.191261] sd 3:0:7:0: [sdh] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 617.191261] sd 3:0:7:0: [sdh] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 617.202791] sd 3:0:9:0: Attached scsi generic sg9 type 0
[ 617.202791] sd 3:0:9:0: Attached scsi generic sg9 type 0
[ 617.203116] sd 3:0:9:0: [sdj] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 617.203116] sd 3:0:9:0: [sdj] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 617.203118] sd 3:0:9:0: [sdj] 4096-byte physical blocks
[ 617.203118] sd 3:0:9:0: [sdj] 4096-byte physical blocks
[ 617.203218] sd 3:0:9:0: [sdj] Write Protect is off
[ 617.203218] sd 3:0:9:0: [sdj] Write Protect is off
[ 617.203347] sd 3:0:9:0: [sdj] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 617.203347] sd 3:0:9:0: [sdj] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 617.209937] sd 3:0:9:0: [sdj] Attached SCSI disk
[ 617.209937] sd 3:0:9:0: [sdj] Attached SCSI disk
[ 617.223990] sd 3:0:7:0: [sdh] Attached SCSI disk
[ 617.223990] sd 3:0:7:0: [sdh] Attached SCSI disk
[ 617.228375] scsi 3:0:10:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 617.228375] scsi 3:0:10:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 617.408424] sd 3:0:10:0: Attached scsi generic sg10 type 0
[ 617.408424] sd 3:0:10:0: Attached scsi generic sg10 type 0
[ 617.408754] sd 3:0:10:0: [sdk] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 617.408754] sd 3:0:10:0: [sdk] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 617.420278] scsi 3:0:11:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 617.420278] scsi 3:0:11:0: Direct-Access HGST HUSMM1640ASS204 C2D0 PQ: 0 ANSI: 6
[ 617.434673] sd 3:0:10:0: [sdk] 4096-byte physical blocks
[ 617.434673] sd 3:0:10:0: [sdk] 4096-byte physical blocks
[ 617.451488] sd 3:0:11:0: Attached scsi generic sg11 type 0
[ 617.451488] sd 3:0:11:0: Attached scsi generic sg11 type 0
[ 617.451838] sd 3:0:11:0: [sdl] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 617.451838] sd 3:0:11:0: [sdl] 781422768 512-byte logical blocks: (400 GB/373 GiB)
[ 617.451840] sd 3:0:11:0: [sdl] 4096-byte physical blocks
[ 617.451840] sd 3:0:11:0: [sdl] 4096-byte physical blocks
[ 617.451944] sd 3:0:11:0: [sdl] Write Protect is off
[ 617.451944] sd 3:0:11:0: [sdl] Write Protect is off
[ 617.452080] sd 3:0:11:0: [sdl] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 617.452080] sd 3:0:11:0: [sdl] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 617.460540] sd 3:0:11:0: [sdl] Attached SCSI disk
[ 617.460540] sd 3:0:11:0: [sdl] Attached SCSI disk
[ 617.461870] sd 3:0:10:0: [sdk] Write Protect is off
[ 617.461870] sd 3:0:10:0: [sdk] Write Protect is off
[ 617.473661] scsi 3:0:12:0: Enclosure 12G SAS Expander RevB PQ: 0 ANSI: 6
[ 617.473661] scsi 3:0:12:0: Enclosure 12G SAS Expander RevB PQ: 0 ANSI: 6
[ 617.499127] ses 3:0:12:0: Attached Enclosure device
[ 617.499127] ses 3:0:12:0: Attached Enclosure device
[ 617.508583] sd 3:0:10:0: [sdk] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 617.508583] sd 3:0:10:0: [sdk] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 617.525816] ses 3:0:12:0: Attached scsi generic sg12 type 13
[ 617.525816] ses 3:0:12:0: Attached scsi generic sg12 type 13
[ 617.542531] sd 3:0:10:0: [sdk] Attached SCSI disk
[ 617.542531] sd 3:0:10:0: [sdk] Attached SCSI disk
root@(none)$
root@(none)$
root@(none)$
root@(none)$
root@(none)$
root@(none)$
root@(none)$
root@(none)$ ./creat_fio_task.sh 4k rw 128
Creat 4k_rw_depth128_fiotest file sucessfully
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
job1: (g=0): rw=rw, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
fio 2.0.5
Starting 12 processes
[ 629.210506] Unable to handle kernel paging request at virtual address 0000ffff8027e048
[ 629.210506] Unable to handle kernel paging request at virtual address 0000ffff8027e048
[ 629.226373] Mem abort info:
[ 629.226373] Mem abort info:
[ 629.231952] ESR = 0x96000006
[ 629.231952] ESR = 0x96000006
[ 629.238052] Exception class = DABT (current EL), IL = 32 bits
[ 629.238052] Exception class = DABT (current EL), IL = 32 bits
[ 629.249898] SET = 0, FnV = 0
[ 629.249898] SET = 0, FnV = 0
[ 629.255998] EA = 0, S1PTW = 0
[ 629.255998] EA = 0, S1PTW = 0
[ 629.262272] Data abort info:
[ 629.262272] Data abort info:
[ 629.268023] ISV = 0, ISS = 0x00000006
[ 629.268023] ISV = 0, ISS = 0x00000006
[ 629.275690] CM = 0, WnR = 0
[ 629.275690] CM = 0, WnR = 0
[ 629.281617] user pgtable: 4k pages, 48-bit VAs, pgdp = 0000000085c91728
[ 629.281617] user pgtable: 4k pages, 48-bit VAs, pgdp = 0000000085c91728
[ 629.294857] [0000ffff8027e048] pgd=00000027a8644003, pud=00000027a85ea003, pmd=0000000000000000
[ 629.294857] [0000ffff8027e048] pgd=00000027a8644003, pud=00000027a85ea003, pmd=0000000000000000
[ 629.312278] Internal error: Oops: 96000006 [#1] PREEMPT SMP
[ 629.312278] Internal error: Oops: 96000006 [#1] PREEMPT SMP
[ 629.323427] Modules linked in: hisi_sas_v3_hw [last unloaded: hisi_sas_v3_hw]
[ 629.323427] Modules linked in: hisi_sas_v3_hw [last unloaded: hisi_sas_v3_hw]
[ 629.337713] CPU: 13 PID: 4465 Comm: fio Not tainted 4.20.0-rc1-15093-ge876dec #1067
[ 629.337713] CPU: 13 PID: 4465 Comm: fio Not tainted 4.20.0-rc1-15093-ge876dec #1067
[ 629.353040] Hardware name: Huawei D06/D06, BIOS Hisilicon D06 UEFI RC0 - B601 (V6.01) 11/08/2018
[ 629.353040] Hardware name: Huawei D06/D06, BIOS Hisilicon D06 UEFI RC0 - B601 (V6.01) 11/08/2018
[ 629.370633] pstate: 80400009 (Nzcv daif +PAN -UAO)
[ 629.370633] pstate: 80400009 (Nzcv daif +PAN -UAO)
[ 629.380218] pc : deadline_remove_request+0x2c/0xd0
[ 629.380218] pc : deadline_remove_request+0x2c/0xd0
[ 629.389799] lr : dd_dispatch_request+0x184/0x1e0
[ 629.389799] lr : dd_dispatch_request+0x184/0x1e0
[ 629.399031] sp : ffff0000275a3aa0
[ 629.399031] sp : ffff0000275a3aa0
[ 629.405651] x29: ffff0000275a3aa0 x28: ffff8027dc230d40
[ 629.405651] x29: ffff0000275a3aa0 x28: ffff8027dc230d40
[ 629.416278] x27: 0000000000000001 x26: ffff8027dc230d40
[ 629.416278] x27: 0000000000000001 x26: ffff8027dc230d40
[ 629.426903] x25: 0000000000000000 x24: ffff0000094f1000
[ 629.426903] x25: 0000000000000000 x24: ffff0000094f1000
[ 629.437529] x23: ffff8027a1e1d140 x22: 0000000000000006
[ 629.437529] x23: ffff8027a1e1d140 x22: 0000000000000006
[ 629.448155] x21: ffff7e009ec69842 x20: ffff8027a1e1d100
[ 629.448155] x21: ffff7e009ec69842 x20: ffff8027a1e1d100
[ 629.458781] x19: ffff8027a1e1d100 x18: 0000000000000000
[ 629.458781] x19: ffff8027a1e1d100 x18: 0000000000000000
[ 629.469406] x17: 000000009e8c9c80 x16: 00000027a3272000
[ 629.469406] x17: 000000009e8c9c80 x16: 00000027a3272000
[ 629.480032] x15: ffff7e009fff9534 x14: ffff7e009ec698c0
[ 629.480032] x15: ffff7e009fff9534 x14: ffff7e009ec698c0
[ 629.490658] x13: ffff7e009ec69880 x12: 0000000000000009
[ 629.490658] x13: ffff7e009ec69880 x12: 0000000000000009
[ 629.501284] x11: 0000000000000000 x10: ffff8027a7c03560
[ 629.501284] x11: 0000000000000000 x10: ffff8027a7c03560
[ 629.511910] x9 : ffff8027a7c007b8 x8 : 0000000000000007
[ 629.511910] x9 : ffff8027a7c007b8 x8 : 0000000000000007
[ 629.522535] x7 : ffff8027a7c011c8 x6 : 0000000000000000
[ 629.522535] x7 : ffff8027a7c011c8 x6 : 0000000000000000
[ 629.533161] x5 : 0000000000000000 x4 : 0000ffff8027e040
[ 629.533161] x5 : 0000000000000000 x4 : 0000ffff8027e040
[ 629.543787] x3 : ffff8027a1e20040 x2 : ffff8027da87f6a0
[ 629.543787] x3 : ffff8027a1e20040 x2 : ffff8027da87f6a0
[ 629.554413] x1 : ffff8027a1e1d140 x0 : ffff8027a1e1d160
[ 629.554413] x1 : ffff8027a1e1d140 x0 : ffff8027a1e1d160
[ 629.565039] Process fio (pid: 4465, stack limit = 0x000000004e5e4c33)
[ 629.565039] Process fio (pid: 4465, stack limit = 0x000000004e5e4c33)
[ 629.577929] Call trace:
[ 629.577929] Call trace:
[ 629.582808] deadline_remove_request+0x2c/0xd0
[ 629.582808] deadline_remove_request+0x2c/0xd0
[ 629.591693] dd_dispatch_request+0x184/0x1e0
[ 629.591693] dd_dispatch_request+0x184/0x1e0
[ 629.600229] blk_mq_do_dispatch_sched+0x4c/0x104
[ 629.600229] blk_mq_do_dispatch_sched+0x4c/0x104
[ 629.609461] blk_mq_sched_dispatch_requests+0x114/0x18c
[ 629.609461] blk_mq_sched_dispatch_requests+0x114/0x18c
[ 629.619913] __blk_mq_run_hw_queue+0xa4/0x11c
[ 629.619913] __blk_mq_run_hw_queue+0xa4/0x11c
[ 629.628624] __blk_mq_delay_run_hw_queue+0x1a4/0x208
[ 629.628624] __blk_mq_delay_run_hw_queue+0x1a4/0x208
[ 629.638553] blk_mq_run_hw_queue+0x9c/0xf8
[ 629.638553] blk_mq_run_hw_queue+0x9c/0xf8
[ 629.646741] blk_mq_sched_insert_requests+0x60/0xac
[ 629.646741] blk_mq_sched_insert_requests+0x60/0xac
[ 629.656495] blk_mq_flush_plug_list+0x144/0x16c
[ 629.656495] blk_mq_flush_plug_list+0x144/0x16c
[ 629.665554] blk_flush_plug_list+0xe8/0x244
[ 629.665554] blk_flush_plug_list+0xe8/0x244
[ 629.673915] blk_finish_plug+0x30/0x40
[ 629.673915] blk_finish_plug+0x30/0x40
[ 629.681408] __arm64_sys_io_submit+0x114/0x218
[ 629.681408] __arm64_sys_io_submit+0x114/0x218
[ 629.690294] el0_svc_common+0x94/0xf0
[ 629.690294] el0_svc_common+0x94/0xf0
[ 629.697611] el0_svc_handler+0x24/0x80
[ 629.697611] el0_svc_handler+0x24/0x80
[ 629.705102] el0_svc+0x8/0x7c0
[ 629.705102] el0_svc+0x8/0x7c0
[ 629.711200] Code: 91010021 f9400ea4 f9402263 f9402662 (f9400496)
[ 629.711200] Code: 91010021 f9400ea4 f9402263 f9402662 (f9400496)
[ 629.723394] ---[ end trace 22423ac28b61ac51 ]---
[ 629.723394] ---[ end trace 22423ac28b61ac51 ]---
[ 629.732631] note: fio[4465] exited with preempt_count 2
[ 629.732631] note: fio[4465] exited with preempt_count 2
[ 629.743112] WARNING: CPU: 9 PID: 4465 at kernel/rcu/tree_plugin.h:337 rcu_note_context_switch+0x38/0x2ec
[ 629.743112] WARNING: CPU: 9 PID: 4465 at kernel/rcu/tree_plugin.h:337 rcu_note_context_switch+0x38/0x2ec
[ 629.762100] Modules linked in: hisi_sas_v3_hw [last unloaded: hisi_sas_v3_hw]
[ 629.762100] Modules linked in: hisi_sas_v3_hw [last unloaded: hisi_sas_v3_hw]
[ 629.776387] CPU: 9 PID: 4465 Comm: fio Tainted: G D 4.20.0-rc1-15093-ge876dec #1067
[ 629.776387] CPU: 9 PID: 4465 Comm: fio Tainted: G D 4.20.0-rc1-15093-ge876dec #1067
[ 629.794327] Hardware name: Huawei D06/D06, BIOS Hisilicon D06 UEFI RC0 - B601 (V6.01) 11/08/2018
[ 629.794327] Hardware name: Huawei D06/D06, BIOS Hisilicon D06 UEFI RC0 - B601 (V6.01) 11/08/2018
[ 629.811919] pstate: 20400089 (nzCv daIf +PAN -UAO)
[ 629.811919] pstate: 20400089 (nzCv daIf +PAN -UAO)
[ 629.821500] pc : rcu_note_context_switch+0x38/0x2ec
[ 629.821500] pc : rcu_note_context_switch+0x38/0x2ec
[ 629.831258] lr : __schedule+0x88/0x5f8
[ 629.831258] lr : __schedule+0x88/0x5f8
Jo[bs: 12 (f=0)
629.838748] sp : ffff0000275a34d0
[ 629.838748] sp : ffff0000275a34d0
[ 629.846670] x29: ffff0000275a34d0 x28: ffff8027dc230d40
[ 629.846670] x29: ffff0000275a34d0 x28: ffff8027dc230d40
[ 629.857296] x27: 0000000000000000 x26: ffff000008da18e0
[ 629.857296] x27: 0000000000000000 x26: ffff000008da18e0
[ 629.867922] x25: ffff0000094f1000 x24: ffff8027dc230d40
[ 629.867922] x25: ffff0000094f1000 x24: ffff8027dc230d40
[ 629.878548] x23: ffff0000094ca000 x22: ffff0000094d6000
[ 629.878548] x23: ffff0000094ca000 x22: ffff0000094d6000
[ 629.889174] x21: ffff8027e0ab3b00 x20: ffff8027dc230d40
[ 629.889174] x21: ffff8027e0ab3b00 x20: ffff8027dc230d40
[ 629.899800] x19: ffff8027e0ab4700 x18: 0000000000000000
[ 629.899800] x19: ffff8027e0ab4700 x18: 0000000000000000
[ 629.910425] x17: 0000000000000000 x16: 0000000000000000
[ 629.910425] x17: 0000000000000000 x16: 0000000000000000
[ 629.921050] x15: 0000000000000000 x14: 0000000000000000
[ 629.921050] x15: 0000000000000000 x14: 0000000000000000
[ 629.931676] x13: 0000000000000000 x12: 0000000000000000
[ 629.931676] x13: 0000000000000000 x12: 0000000000000000
[ 629.942301] x11: 0000000000000002 x10: 0000000000000970
[ 629.942301] x11: 0000000000000002 x10: 0000000000000970
[ 629.952927] x9 : ffff0000275a3660 x8 : ffff8027dc231710
[ 629.952927] x9 : ffff0000275a3660 x8 : ffff8027dc231710
[ 629.963553] x7 : 000000927fe91094 x6 : ffff8027a0308428
[ 629.963553] x7 : 000000927fe91094 x6 : ffff8027a0308428
[ 629.974178] x5 : 0000000000000000 x4 : 0000000000000001
[ 629.974178] x5 : 0000000000000000 x4 : 0000000000000001
[ 629.984804] x3 : 674238a07d45b500 x2 : 00008027d75dd000
[ 629.984804] x3 : 674238a07d45b500 x2 : 00008027d75dd000
[ 629.995429] x1 : 00008027d75dd000 x0 : 0000000000000001
[ 629.995429] x1 : 00008027d75dd000 x0 : 0000000000000001
[ 630.006055] Call trace:
[ 630.006055] Call trace:
[ 630.010934] rcu_note_context_switch+0x38/0x2ec
[ 630.010934] rcu_note_context_switch+0x38/0x2ec
[ 630.019993] __schedule+0x88/0x5f8
[ 630.019993] __schedule+0x88/0x5f8
[ 630.026788] schedule+0x38/0x9c
[ 630.026788] schedule+0x38/0x9c
[ 630.033060] schedule_timeout+0x1b8/0x320
[ 630.033060] schedule_timeout+0x1b8/0x320
[ 630.041074] wait_for_common+0xb4/0x164
[ 630.041074] wait_for_common+0xb4/0x164
[ 630.048739] wait_for_completion+0x14/0x1c
[ 630.048739] wait_for_completion+0x14/0x1c
[ 630.056928] exit_aio+0x100/0x114
[ 630.056928] exit_aio+0x100/0x114
[ 630.063549] mmput+0x48/0x118
[ 630.063549] mmput+0x48/0x118
[ 630.069473] do_exit+0x280/0x984
[ 630.069473] do_exit+0x280/0x984
[ 630.075920] die+0x1e0/0x20c
[ 630.075920] die+0x1e0/0x20c
[ 630.081670] die_kernel_fault+0x64/0x74
[ 630.081670] die_kernel_fault+0x64/0x74
[ 6Jobs: 12 (f=0)
30.089335] __do_kernel_fault+0x84/0xac
[ 630.089335] __do_kernel_fault+0x84/0xac
[ 630.098477] do_page_fault+0x114/0x3a8
[ 630.098477] do_page_fault+0x114/0x3a8
[ 630.105968] do_translation_fault+0x68/0x74
[ 630.105968] do_translation_fault+0x68/0x74
[ 630.114330] do_mem_abort+0x3c/0x9c
[ 630.114330] do_mem_abort+0x3c/0x9c
[ 630.121298] el1_da+0x20/0x80
[ 630.121298] el1_da+0x20/0x80
[ 630.127223] deadline_remove_request+0x2c/0xd0
[ 630.127223] deadline_remove_request+0x2c/0xd0
[ 630.136108] dd_dispatch_request+0x184/0x1e0
[ 630.136108] dd_dispatch_request+0x184/0x1e0
[ 630.144644] blk_mq_do_dispatch_sched+0x4c/0x104
[ 630.144644] blk_mq_do_dispatch_sched+0x4c/0x104
[ 630.153877] blk_mq_sched_dispatch_requests+0x114/0x18c
[ 630.153877] blk_mq_sched_dispatch_requests+0x114/0x18c
[ 630.164329] __blk_mq_run_hw_queue+0xa4/0x11c
[ 630.164329] __blk_mq_run_hw_queue+0xa4/0x11c
[ 630.173040] __blk_mq_delay_run_hw_queue+0x1a4/0x208
[ 630.173040] __blk_mq_delay_run_hw_queue+0x1a4/0x208
[ 630.182969] blk_mq_run_hw_queue+0x9c/0xf8
[ 630.182969] blk_mq_run_hw_queue+0x9c/0xf8
[ 630.191157] blk_mq_sched_insert_requests+0x60/0xac
[ 630.191157] blk_mq_sched_insert_requests+0x60/0xac
[ 630.200911] blk_mq_flush_plug_list+0x144/0x16c
[ 630.200911] blk_mq_flush_plug_list+0x144/0x16c
[ 630.209970] blk_flush_plug_list+0xe8/0x244
[ 630.209970] blk_flush_plug_list+0xe8/0x244
[ 630.218331] blk_finish_plug+0x30/0x40
[ 630.218331] blk_finish_plug+0x30/0x40
[ 630.225823] __arm64_sys_io_submit+0x114/0x218
[ 630.225823] __arm64_sys_io_submit+0x114/0x218
[ 630.234708] el0_svc_common+0x94/0xf0
[ 630.234708] el0_svc_common+0x94/0xf0
[ 630.242025] el0_svc_handler+0x24/0x80
[ 630.242025] el0_svc_handler+0x24/0x80
[ 630.249516] el0_svc+0x8/0x7c0
[ 630.249516] el0_svc+0x8/0x7c0
[ 630.255613] ---[ end trace 22423ac28b61ac52 ]---
[ 630.255613] ---[ end trace 22423ac28b61ac52 ]---
