RE: fio-2c2c93d and with a lot of numjobs causing fio Jobs to Time Out

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Jens and Andrey,

>I'll be taking a bit of time off, but I should have access to an AIX
>that I can test on sometime next week. What version are you on?

At this time i use a VIOS 2.2.3.52  aka AIX 6100-08-01-1245. I need to build fio against AIX 6.x, 7.x. 
After next week, i'll reinstall the server to go to production. So i'll need to look for a new box with a FC card so i'll use another AIX version.

For your information i don't have the issue with:
HDISK=hdisk6 RWTYPE=rw RRATIO=0 BS=4096 QD=128 NJ=128  /home/padmin/fio/fio-5f3bd0f/fio myfio.fio
but
HDISK=hdisk6 RWTYPE=rw RRATIO=0 BS=65536 QD=128 NJ=128  /home/padmin/fio/fio-5f3bd0f/fio myfio.fio

So i think it's perhaps some hardware or OS comportement that the cause of the issue it's not fio fio  (it's not the guilty but a collateral victim) but idon't find why because the statistics data from SAN seems OK.
SAN stats during test:
Latency (ms):      R: 0/ W 0.17 
Thoughput (M): R 0  / W 9.8
IOPS:                    R 0 / W 2380


1. fIo stay stuck with the last print:
   fio: job 'test' hasn't exited in 97 seconds, it appears to be stuck. Doing forceful exit of this job.

1.1 The only process that's stay a live is the min fio process (all others 128 childs have been died or finished)
# ps -ef | grep fio | grep -v grep
    root 41877588 48234628 120 16:18:23  pts/1  1:20 /home/padmin/fio/fio-5f3bd0f/fio myfio.fio
#

1.3 System monitor indicate no disk activty test is over but the main process uses CPU.
   Topas Monitor for host:    XVIOS16           
   Thu Dec 31 16:28:19 2015   Interval:  2      
                                                
   CPU  User%  Kern%  Wait%  Idle%  Physc   Entc
   ALL   57.4    5.8    0.0   36.8   1.01  101.5

   Disk    Busy%     KBPS     TPS KB-Read KB-Writ
   hdisk3    0.0     0.0      0.0    0.0     0.0 
   hdisk2    0.0     0.0      0.0    0.0     0.0 
   hdisk9    0.0     0.0      0.0    0.0     0.0 
   hdisk1    0.0     0.0      0.0    0.0     0.0 
   hdisk0    0.0     0.0      0.0    0.0     0.0 
   hdisk10   0.0     0.0      0.0    0.0     0.0 
   hdisk7    0.0     0.0      0.0    0.0     0.0 
   hdisk4    0.0     0.0      0.0    0.0     0.0 
   hdisk5    0.0     0.0      0.0    0.0     0.0 
   hdisk6    0.0     0.0      0.0    0.0     0.0 

   Name            PID  CPU%  PgSp Owner
   fio        41877588  62.6   3.5 root 
   sched        196614   0.1   0.4 root
   vio_daem   10682558   0.0  46.1 root
   seaproc     3997822   0.0   1.0 root
   syncd       3538998   0.0   0.6 root
   java        9764964   0.0  84.2 root

1.5 During gdb against the process the cpu load disapear. If i don't dgb against the process the load stay alive.

2. 1:  gdb trace
   # gdb -p 41877588
   GNU gdb (GDB) 7.8.1
   Copyright (C) 2014 Free Software Foundation, Inc.
   License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
   This is free software: you are free to change and redistribute it.
   There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
   and "show warranty" for details.
   This GDB was configured as "powerpc-ibm-aix5.1.0.0".
   Type "show configuration" for configuration details.
   For bug reporting instructions, please see:
   <http://www.gnu.org/software/gdb/bugs/>.
   Find the GDB manual and other documentation resources online at:
   <http://www.gnu.org/software/gdb/documentation/>.
   For help, type "help".
   Type "apropos word" to search for commands related to "word".
   Attaching to process 41877588
   0xd0119598 in ?? ()
   (gdb) info threads
     Id   Target Id         Frame
   * 1    process 41877588  0xd0119598 in ?? ()
   (gdb) thread 1
   [Switching to thread 1 (process 41877588)]
   #0  0xd0119598 in ?? ()
   (gdb) bt
   #0  0xd0119598 in ?? ()
   #1  0xffffffff in ?? ()
   #2  0xd0119bc0 in ?? ()
   #3  0xd0118f28 in ?? ()
   #4  0x100625dc in ?? ()
   #5  0x1006482c in ?? ()
   #6  0x10064b90 in ?? ()
   #7  0xd0505c10 in ?? ()
   #8  0x00000000 in ?? ()
   (gdb)

2.1.1 It semmes i nned to build fio with some option to have some symbole. Which option i can use for that.
At this time i just do: ./configure; gmake CC=gcc

2.2: The full log from fio:
   # HDISK=hdisk6 RWTYPE=rw RRATIO=0 BS=65536 QD=128 NJ=128  /home/padmin/fio/fio-5f3bd0f/fio myfio.fio
   test: (g=0): rw=rw, bs=64K-64K/64K-64K/64K-64K, ioengine=posixaio, iodepth=128
   ...
   fio-2.3
   Starting 128 processes
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.:22s]
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job. iops] [eta 11m:34s]
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.] [0/138/0 iops] [eta 17m:34s]
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.4KB/0KB /s] [0/215/0 iops] [eta 29m:36s]
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.4.6% done] [0KB/7488KB/0KB /s] [0/117/0 iops] [eta 35m:36s]
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.1),W(21)] [12.8% done] [0KB/8617KB/0KB /s] [0/134/0 iops] [eta 41m:40s]
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.1),W(38)] [11.4% done] [0KB/856KB/0KB /s] [0/13/0 iops] [eta 47m:44s]
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 118 (f=118): [f(1),W(8),_(1),W(11),_(1),W(2),f(1),W(5),_(1),W(9),_(1),W(7),_(1),W(12),_(1),W(6),_(1),W(11),_(1),W(2),_(1),W(5),_(1),W(38)] [9.3% done] [0KB/20057KB/0KB fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 117 (f=117): [W(9),_(1),W(11),_(1),W(8),_(1),W(9),_(1),W(4),f(1),W(2),_(1),W(1),_(1),W(10),_(1),W(6),_(1),W(11),_(1),W(2),_(1),W(5),_(1),W(38)] [8.5% done] [0KB/7255KB/fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 116 (f=116): [W(3),_(1),W(5),_(1),W(11),_(1),W(8),_(1),W(9),_(1),W(7),_(1),W(1),_(1),W(10),_(1),W(6),_(1),W(11),_(1),W(2),_(1),W(5),_(1),W(38)] [7.9% done] [0KB/1130KB/Jobs: 116 (f=116): [W(2),f(1),_(1),W(5),_(1),W(2),f(1),W(8),_(1),W(8),_(1),W(4),f(1),W(4),_(1),W(7),_(1),W(1),_(1),W(10),_(1),W(6),_(1),W(11),_(1),W(2),_(1),W(5),_(1),W(38)] Jobs: 116 (f=116): [W(3),_(1),W(5),_(1),W(1),f(1),W(9),_(1),W(8),_(1),W(4),f(1),W(4),_(1),W(7),_(1),W(1),_(1),W(10),_(1),W(6),_(1),W(11),_(1),W(2),_(1),W(5),_(1),W(38)] [7.9%Jobs: 116 (f=116): [W(3),_(1),W(5),_(1),W(11),_(1),W(8),_(1),W(9),_(1),W(7),_(1),W(1),_(1),W(10),_(1),W(6),_(1),W(11),_(1),W(2),_(1),W(5),_(1),W(38)] [7.9% done] [0KB/0KB/0KBfio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 115 (f=115): [W(3),_(1),W(5),_(1),W(11),_(1),W(3),f(1),W(4),_(1),W(9),_(1),W(7),_(1),W(1),_(1),W(1),f(1),W(8),_(1),W(6),_(1),W(11),_(1),W(2),_(1),W(5),_(1),W(36),_(1),WJobs: 115 (f=115): [W(3),_(1),W(5),_(1),W(11),_(1),W(8),_(1),W(9),_(1),W(7),_(1),W(1),_(1),W(10),_(1),W(6),_(1),W(6),f(1),W(4),_(1),W(2),_(1),W(5),_(1),W(36),_(1),W(1)] [7.4%fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 114 (f=114): [W(3),_(1),W(5),_(1),W(4),f(1),W(6),_(1),W(8),_(1),W(9),_(1),W(7),_(1),W(1),_(1),W(10),_(1),W(6),_(1),W(11),_(1),W(2),_(1),W(3),_(1),W(1),_(1),W(36),_(1),Wfio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 113 (f=113): [W(3),_(1),W(5),_(1),W(11),_(1),W(6),f(1),W(1),_(1),W(9),_(1),W(7),_(1),W(1),_(1),W(10),_(1),W(6),_(1),W(11),_(1),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),WJobs: 113 (f=113): [W(3),_(1),W(5),_(1),W(11),_(1),W(6),f(1),W(1),_(1),W(2),f(2),W(5),_(1),W(7),_(1),W(1),_(1),f(1),W(9),_(1),W(6),_(1),W(11),_(1),W(2),_(1),W(1),_(1),W(1),_(Jobs: 113 (f=113): [W(3),_(1),W(5),_(1),W(11),_(1),W(8),_(1),W(2),f(2),W(5),_(1),W(7),_(1),W(1),_(1),f(1),W(9),_(1),W(6),_(1),W(11),_(1),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 111 (f=111): [W(3),_(1),W(5),_(1),W(11),_(1),W(7),f(1),_(1),W(9),_(2),W(6),_(1),W(1),_(1),W(3),_(1),W(6),_(1),W(6),_(1),W(11),_(1),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 110 (f=110): [W(3),_(1),W(5),_(1),W(10),f(1),_(1),W(8),_(1),W(9),_(2),W(6),_(1),W(1),_(1),W(3),_(1),W(6),_(1),W(6),_(1),W(11),_(1),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(Jobs: 109 (f=109): [W(3),_(1),f(1),W(4),_(1),W(10),f(1),_(1),W(8),_(1),W(9),_(2),W(6),_(1),W(1),_(1),W(3),_(1),W(6),_(1),W(6),_(1),W(11),_(1),W(2),_(1),W(1),_(1),W(1),_(1),W(fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.:55m:38s]
   Jobs: 108 (f=108): [W(3),_(1),f(1),W(2),f(1),W(1),_(1),W(7),f(1),W(3),_(1),W(8),_(1),W(9),_(2),W(6),_(1),W(1),_(1),W(3),_(1),W(2),f(1),W(3),_(1),W(6),_(1),W(8),_(1),W(2),_(1)fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job./0KB /s] [0/142/0 iops] [eta 02h:01m:37s]
   Jobs: 107 (f=107): [W(3),_(1),W(5),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(6),_(1),W(1),_(1),W(3),_(1),W(6),_(1),W(6),_(1),W(8),_(1),W(2),_(1),W(2),_(1),W(1),_(1),W(1Jobs: 107 (f=107): [W(3),_(1),W(5),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(6),_(1),W(1),_(1),W(3),_(1),W(6),_(1),W(6),_(1),W(8),_(1),W(2),_(1),W(2),_(1),W(1),_(1),W(1fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.eta 02h:07m:35s]
   Jobs: 106 (f=106): [W(3),_(1),W(5),_(1),W(6),f(1),W(1),f(1),W(2),_(1),W(5),_(1),W(2),_(1),W(7),f(1),W(1),_(2),W(6),_(1),W(1),_(1),W(3),_(1),W(6),_(1),W(6),_(1),W(8),_(1),W(2)Jobs: 106 (f=106): [W(3),_(1),W(5),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(6),_(1),W(1),_(1),W(3),_(1),W(6),_(1),W(6),_(1),W(8),_(1),W(2),_(1),W(2),_(1),W(1),_(1),W(1fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.42/0 iops] [eta 02h:13m:57s]
   Jobs: 105 (f=105): [W(3),_(1),W(5),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(6),_(1),W(1),_(1),W(3),_(2),W(5),_(1),W(6),_(1),W(8),_(1),W(2),_(1),W(2),_(1),W(1),_(1),W(1Jobs: 105 (f=105): [W(3),_(1),W(5),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(6),_(1),W(1),_(1),W(3),_(2),W(2),f(1),W(2),_(1),W(6),_(1),W(8),_(1),W(2),_(1),W(2),_(1),W(1fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.3/0 iops] [eta 02h:19m:56s]
   Jobs: 104 (f=104): [W(3),_(1),W(5),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(6),_(1),W(1),_(1),W(3),_(2),W(2),f(1),W(2),_(1),W(3),_(1),W(2),_(1),W(8),_(1),W(2),_(1),W(1fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.B/0KB /s] [0/136/0 iops] [eta 02h:26m:24s]
   Jobs: 103 (f=103): [W(3),_(1),W(5),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(6),_(1),W(1),_(1),W(3),_(2),W(5),_(1),W(3),_(1),W(2),_(1),W(8),_(1),W(2),_(1),W(1),f(1),_(1Jobs: 103 (f=103): [W(3),_(1),W(5),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(6),_(1),W(1),_(1),W(3),_(2),W(5),_(1),W(3),_(1),W(2),_(1),W(8),_(1),W(2),_(1),W(2),_(1),W(1fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.ops] [eta 02h:32m:22s]
   Jobs: 103 (f=103): [W(3),_(1),W(5),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(6),_(1),W(1),_(1),W(3),_(2),W(5),_(1),W(2),f(1),_(1),W(2),_(1),W(8),_(1),W(2),_(1),W(2),_(1fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job. [0/205/0 iops] [eta 02h:38m:22s]
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 97 (f=97): [W(3),_(1),W(5),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(6),_(3),W(3),_(2),W(5),_(1),W(3),_(1),W(2),_(2),W(7),_(1),W(2),_(1),W(2),_(1),W(1),_(1),W(1),Jobs: 97 (f=97): [W(3),_(1),W(5),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(6),_(3),W(3),_(2),W(5),_(1),W(3),_(1),W(2),_(2),W(6),f(1),_(1),W(2),_(1),W(2),_(1),W(1),_(1),Jobs: 97 (f=97): [W(3),_(1),W(5),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(6),_(3),W(3),_(2),W(5),_(1),W(3),_(1),W(2),_(2),W(7),_(1),W(2),_(1),W(2),_(1),W(1),_(1),W(1),fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job./0KB /s] [0/0/0 iops] [eta 03h:08m:26s]
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 94 (f=94): [W(3),_(1),W(4),f(1),_(1),W(11),_(1),W(5),_(1),W(2),_(1),f(1),W(8),_(2),W(6),_(3),W(3),_(2),W(5),_(1),W(3),_(1),W(2),_(2),W(7),_(1),W(2),_(1),W(2),_(1),W(1),fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job. done] [0KB/14019KB/0KB /s] [0/219/0 iops] [eta 03h:26m:33s]
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 92 (f=92): [W(3),_(1),W(4),f(1),_(1),W(11),_(1),W(5),_(1),W(2),_(1),f(1),W(8),_(2),W(6),_(3),W(3),_(2),W(5),_(1),W(3),_(1),W(2),_(2),W(7),_(1),W(2),_(1),W(2),_(1),W(1),fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.(2)] [2.9% done] [0KB/1593KB/0KB /s] [0/24/0 iops] [eta 03h:38m:38s]
   Jobs: 91 (f=91): [W(3),_(1),W(5),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(6),_(3),W(3),_(2),W(5),_(1),W(3),_(1),W(2),_(2),W(7),_(1),W(2),_(1),W(2),_(1),W(1),_(1),W(1),fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.(2)] [2.9% done] [0KB/5694KB/0KB /s] [0/88/0 iops] [eta 03h:44m:49s]
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 89 (f=89): [W(3),_(1),W(5),_(1),W(11),_(1),W(4),f(1),_(1),W(2),_(1),W(6),f(1),W(2),_(2),W(6),_(3),W(3),_(3),W(4),_(1),W(3),_(1),W(2),_(2),W(1),f(1),W(5),_(1),W(2),_(1),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(3),W(3),_(1),f(1),_(1),W(1),_(1),W(5),_(1),W(1),_(2),W(3),_(2),W(2),_(3),W(4),_(1),W(2),_(2)] [2.7% done] [0KB/15730KB/0KB /s] [0/245/0 ifio: job 'test' hasn't exited in 61 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 87 (f=87): [W(3),_(1),W(5),_(1),W(11),_(1),W(4),f(1),_(1),W(2),_(1),W(6),f(1),W(2),_(2),W(6),_(3),W(3),_(3),W(4),_(1),W(3),_(1),W(2),_(2),W(1),f(1),W(5),_(1),W(2),_(2),W(1),_(1),W(1),_(3),W(1),_(3),W(2),f(1),_(1),f(1),_(1),W(1),_(1),W(5),_(1),W(1),_(2),W(3),_(2),W(2),_(3),W(4),_(1),W(2),_(2)] [2.6% done] [0KB/12825KB/0KB /s] [0/200/0 iops] fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 94 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 85 (f=85): [W(3),_(1),W(5),_(1),W(11),_(1),W(4),f(1),_(1),W(2),_(1),W(9),_(2),W(6),_(3),W(3),_(3),W(4),_(1),W(1),_(1),W(1),_(1),W(2),_(2),W(1),f(1),W(5),_(1),W(2),_(2),W(1),_(1),W(1),_(3),W(1),_(3),f(1),W(2),_(1),W(1),_(1),W(1),_(1),W(5),_(1),W(1),_(2),W(3),_(2),W(2),_(3),W(2),_(1),W(1),_(1),W(2),_(2)] [2.5% done] [0KB/10149KB/0KB /s] [0/15fio: job 'test' hasn't exited in 100 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 86 (f=86): [W(3),_(1),W(5),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(2),f(1),W(3),_(3),W(3),_(3),W(4),_(1),W(1),_(1),W(1),_(1),W(2),_(2),W(7),_(1),W(2),f(1),_(1),W(1),_(1),W(1),_(3),W(1),_(3),f(1),W(2),_(1),W(1),_(1),W(1),_(1),W(5),_(1),W(1),_(2),W(3),_(2),W(2),_(3),W(2),_(1),W(1),_(1),W(2),_(2)] [2.4% done] [0KB/15226KB/0KB /s] [0/23fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 103 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 83 (f=83): [W(1),_(1),W(1),_(1),W(5),_(1),W(9),f(1),W(1),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(6),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(2),W(7),_(1),W(2),_(2),W(1),_(1),W(1),_(3),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(1),W(5),_(1),W(1),_(2),W(3),_(2),W(2),_(3),W(2),_(1),W(1),_(1),W(2),_(2)] [2.2% done] [0KB/12140KB/0KB fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 103 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 81 (f=81): [W(1),_(1),W(1),_(1),W(1),_(1),W(3),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(4),_(1),W(1),_(3),W(3),_(3),W(2),_(1),f(1),_(1),W(1),_(1),W(1),_(1),W(2),_(2),W(4),f(1),W(2),_(1),W(2),_(2),W(1),_(1),W(1),_(3),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(1),W(5),_(1),W(1),_(2),W(3),_(2),W(2),_(3),W(2),_(1),W(1),_(1),W(2),_(2)] [2.1% dofio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 81 (f=81): [W(1),_(1),W(1),_(1),W(1),_(1),W(3),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(4),_(1),W(1),_(3),W(3),_(3),W(2),_(1),f(1),_(1),W(1),_(1),W(1),_(1),f(1),W(1),_(2),W(7),_(1),W(2),_(2),W(1),_(1),W(1),_(3),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(1),W(5),_(1),W(1),_(2),W(3),_(2),W(2),_(3),W(2),_(1),W(1),_(1),W(2),_(2)] [2.1% done] [fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 77 (f=77): [W(1),_(1),W(1),_(4),W(2),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(4),_(1),W(1),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),f(1),W(1),_(2),W(7),_(1),W(2),_(2),W(1),_(1),W(1),_(3),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(3),W(3),_(1),W(1),_(2),W(3),_(2),W(2),_(3),W(2),_(1),W(1),_(1),W(2),_(2)] [1.9% done] [0KB/275KB/fio: job 'test' hasn't exited in 89 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 77 (f=77): [W(1),_(1),W(1),_(4),W(2),_(1),W(11),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(4),_(1),f(1),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(2),W(7),_(1),W(2),_(2),W(1),_(1),W(1),_(3),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(3),W(3),_(1),W(1),_(2),W(3),_(2),W(2),_(3),W(2),_(1),W(1),_(1),W(2),_(2)] [1.9% done] [0KB/16256KB/0KBfio: job 'test' hasn't exited in 104 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 77 (f=77): [W(1),_(1),W(1),_(4),W(2),_(1),W(5),f(1),W(5),_(1),W(5),_(1),W(2),_(1),W(9),_(2),W(4),_(1),f(1),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(2),W(7),_(1),W(2),_(2),W(1),_(1),W(1),_(3),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(3),W(3),_(1),W(1),_(2),W(3),_(2),W(2),_(3),W(2),_(1),W(1),_(1),W(2),_(2)] [1.9% done] [0KB/29fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 110 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 75 (f=75): [W(1),_(1),W(1),_(4),W(2),_(1),W(5),f(1),W(5),_(1),W(5),_(1),W(2),_(1),W(1),_(1),W(7),_(2),f(1),W(3),_(1),W(1),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(2),W(7),_(1),W(2),_(2),W(1),_(5),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(3),W(3),_(1),W(1),_(2),W(3),_(2),W(2),_(3),W(2),_(1),W(1),_(1),W(2),_(2)] [1.8% done] [0fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 73 (f=73): [W(1),_(1),W(1),_(4),W(2),_(1),W(11),_(2),W(4),_(1),W(2),_(1),W(1),_(1),W(7),_(2),W(4),_(1),W(1),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(2),W(2),f(1),W(4),_(1),W(2),_(2),W(1),_(5),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(3),W(3),_(4),W(3),_(2),W(2),_(3),W(2),_(1),W(1),_(1),W(2),_(2)] [1.7% done] [0KB/8633KB/0KB fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 71 (f=71): [W(1),_(1),W(1),_(4),W(2),_(1),W(1),_(1),W(1),f(1),W(7),_(2),W(4),_(1),W(2),_(1),W(1),_(1),W(7),_(2),W(4),_(1),W(1),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(2),W(7),_(1),W(2),_(2),W(1),_(5),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(3),W(3),_(4),W(3),_(2),W(1),_(4),W(2),_(1),W(1),_(1),W(2),_(2)] [1.7% done] [0KB/51fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 69 (f=69): [W(1),_(1),W(1),_(4),W(2),_(1),W(1),_(2),W(8),_(2),W(4),_(1),W(2),_(1),W(1),_(1),W(7),_(2),W(4),_(1),W(1),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(2),W(7),_(2),W(1),_(2),W(1),_(5),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(3),W(3),_(4),f(1),W(2),_(2),W(1),_(4),W(2),_(1),W(1),_(1),f(1),W(1),_(2)] [1.6% done] [0KB/14Jobs: 69 (f=69): [W(1),_(1),W(1),_(4),W(2),_(1),W(1),_(2),W(8),_(2),W(4),_(1),W(2),_(1),W(1),_(1),W(7),_(2),W(4),_(1),W(1),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(2),W(3),f(1),W(3),_(2),W(1),_(2),W(1),_(5),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(3),W(3),_(4),f(1),W(2),_(2),W(1),_(4),W(2),_(1),W(1),_(1),f(1),W(1),_(2)] [1.6% donfio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 68 (f=68): [W(1),_(1),W(1),_(4),W(2),_(1),W(1),_(2),W(8),_(2),W(1),_(1),W(2),_(1),W(2),_(1),W(1),_(1),W(7),_(2),W(1),f(1),W(2),_(1),W(1),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(2),W(3),f(1),W(3),_(2),W(1),_(2),W(1),_(5),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(3),W(3),_(4),W(3),_(2),W(1),_(4),W(2),_(1),W(1),_(1),W(2),_(2)]Jobs: 67 (f=67): [W(1),_(1),W(1),_(4),W(2),_(1),W(1),_(2),W(1),_(1),W(6),_(2),W(1),_(1),W(2),_(1),W(2),_(1),W(1),_(1),W(7),_(2),W(4),_(1),W(1),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(2),W(7),_(2),W(1),_(2),W(1),_(5),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(3),W(1),f(1),W(1),_(4),W(3),_(2),W(1),_(4),W(2),_(1),W(1),_(1),W(2),_(2)]fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 100 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 64 (f=64): [_(2),W(1),_(4),W(2),_(1),W(1),_(2),W(1),_(1),W(6),_(2),W(1),_(1),W(2),_(1),W(2),_(1),W(1),_(1),W(7),_(2),W(3),_(2),W(1),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(1),f(1),_(3),W(6),_(2),f(1),_(2),W(1),_(5),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(3),W(1),f(1),W(1),_(4),W(3),_(2),W(1),_(4),f(1),W(1),_(1),W(1),_(1),W(2),Jobs: 63 (f=63): [_(7),W(2),_(1),W(1),_(2),W(1),_(1),W(6),_(2),W(1),_(1),W(2),_(1),W(2),_(1),W(1),_(1),W(7),_(2),W(3),_(2),W(1),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(1),f(1),_(3),W(6),_(2),f(1),_(2),W(1),_(5),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(3),W(3),_(4),W(3),_(2),W(1),_(4),f(1),W(1),_(1),W(1),_(1),W(2),_(2)] [1.5% done] [0fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 62 (f=62): [_(7),W(2),_(1),W(1),_(2),W(1),_(1),W(6),_(2),W(1),_(1),W(2),_(2),W(1),_(1),W(1),_(1),W(7),_(2),W(3),_(2),W(1),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(3),W(6),_(2),W(1),_(2),W(1),_(5),W(1),_(3),W(3),_(1),W(1),_(1),W(1),_(3),W(3),_(4),W(3),_(2),W(1),_(4),W(2),_(1),W(1),_(1),W(2),_(2)] [1.5% done] [0KB/1263KB/Jobs: 60 (f=60): [_(7),W(2),_(1),W(1),_(2),W(1),_(1),W(6),_(2),W(1),_(1),W(2),_(2),W(1),_(1),W(1),_(1),W(2),_(1),W(4),_(2),W(3),_(2),W(1),_(3),W(3),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(3),W(6),_(2),W(1),_(2),W(1),_(5),W(1),_(3),W(1),_(1),W(1),_(1),W(1),_(1),W(1),_(3),W(3),_(4),W(2),f(1),_(2),W(1),_(4),W(2),_(1),W(1),_(1),W(2),_(2)]fio: job 'test' hasn't exited in 101 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 59 (f=59): [_(7),W(2),_(1),W(1),_(2),W(1),_(1),W(6),_(2),W(1),_(1),W(2),_(2),W(1),_(1),W(1),_(1),W(2),_(1),W(4),_(2),W(3),_(2),W(1),_(3),W(2),f(1),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(3),W(4),_(1),W(1),_(2),W(1),_(2),W(1),_(5),W(1),_(3),W(1),_(1),W(1),_(1),W(1),_(1),W(1),_(3),W(3),_(4),W(3),_(2),W(1),_(4),W(2),_(1),W(1),_(1),fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 58 (f=58): [_(7),W(2),_(1),W(1),_(2),W(1),_(1),W(6),_(2),W(1),_(1),W(2),_(2),W(1),_(1),W(1),_(2),W(1),_(1),W(4),_(2),W(3),_(2),W(1),_(3),W(2),f(1),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(3),W(4),_(1),W(1),_(2),W(1),_(2),W(1),_(5),W(1),_(3),W(1),_(1),W(1),_(1),W(1),_(1),W(1),_(3),W(3),_(4),W(1),f(1),W(1),_(2),W(1),_(4),W(2),_(1),fio: job 'test' hasn't exited in 60 seconds, it appears to be stuck. Doing forceful exit of this job.
   Jobs: 56 (f=56): [_(7),W(2),_(1),W(1),_(2),W(1),_(1),W(6),_(2),f(1),_(2),W(1),_(2),W(1),_(1),W(1),_(2),W(1),_(1),W(3),f(1),_(2),W(3),_(2),W(1),_(3),W(1),_(1),f(1),_(3),W(2),_(1),W(1),_(1),W(1),_(1),W(1),_(1),W(2),_(3),W(4),_(1),W(1),_(2),W(1),_(2),W(1),_(5),W(1),_(3),W(1),_(1),W(1),_(1),W(1),_(1),W(1),_(3),f(1),W(2),_(4),W(3),_(2),W(1),_(4),W(2),fio: job 'test' hasn't exited in 92 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 95 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 96 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 100 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 95 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 95 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 95 seconds, it appears to be stuck. Doing forceful exit of this job.
   fio: job 'test' hasn't exited in 97 seconds, it appears to be stuck. Doing forceful exit of this job.

Thanks for your feedback

Regards,
Thierry

--
To unsubscribe from this list: send the line "unsubscribe fio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [Linux Kernel]     [Linux SCSI]     [Linux IDE]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux SCSI]

  Powered by Linux