Thanks Jens
I did :
$ git pull
$ make clean
$ ./configure
$ make
$ sudo make install
then :
#to remove old files
$ rm -rf /mnt/vda2/tmp/*
# causing the new fio to lay the files out
$ fio -output output-fio-2.1-8-gb2c77.txt four-threads-randio-016.fio
# then reusing them for the old fio version
$ /usr/bin/fio -output output-fio-2.0.13.txt four-threads-randio-016.fio
and I reproduced the problem on my laptop
[ecomar@edow500 devel]$ uname -a
Linux edow500.hursley.ibm.com 2.6.32-358.6.1.el6.x86_64 #1 SMP Fri Mar 29
16:51:51 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux
full results below
HTH
Edo
============================================================================
#config-host.h
/*
* Automatically generated by configure - do not modify
* Configured with: * './configure'
*/
#define CONFIG_64BIT
#define CONFIG_LITTLE_ENDIAN
#define CONFIG_ZLIB
#define CONFIG_LIBAIO
#define CONFIG_POSIXAIO
#define CONFIG_POSIXAIO_FSYNC
#define CONFIG_LINUX_FALLOCATE
#define CONFIG_POSIX_FALLOCATE
#define CONFIG_FDATASYNC
#define CONFIG_SYNC_FILE_RANGE
#define CONFIG_SFAA
#define CONFIG_CLOCK_GETTIME
#define CONFIG_CLOCK_MONOTONIC
#define CONFIG_GETTIMEOFDAY
#define CONFIG_POSIX_FADVISE
#define CONFIG_3ARG_AFFINITY
#define CONFIG_STRSEP
#define CONFIG_STRCASESTR
#define CONFIG_GETOPT_LONG_ONLY
#define CONFIG_INET_ATON
#define CONFIG_SOCKLEN_T
#define CONFIG_LINUX_EXT4_MOVE_EXTENT
#define CONFIG_LINUX_SPLICE
#define CONFIG_TLS_THREAD
#define CONFIG_RUSAGE_THREAD
#define CONFIG_SCHED_IDLE
#define CONFIG_TCP_NODELAY
#define CONFIG_RLIMIT_MEMLOCK
#define CONFIG_PWRITEV
============================================================================
#cat four-threads-randio-016.fio
[global]
size=256m
directory=/mnt/vda2/tmp
invalidate=1
runtime=60
time_based
ramp_time=5
clat_percentiles=0
direct=1
[bgwriter]
rw=randwrite
iodepth=32
ioengine=libaio
[queryA]
rw=randread
ioengine=mmap
thinktime=3
[queryB]
rw=randread
ioengine=mmap
thinktime=5
[bgupdater]
rw=randrw
ioengine=libaio
iodepth=16
size=64m
thinktime=5
============================================================================
bgwriter: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio,
iodepth=32
queryA: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=mmap, iodepth=1
queryB: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=mmap, iodepth=1
bgupdater: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio,
iodepth=16
fio-2.1-8-gb2c77
Starting 4 processes
bgwriter: Laying out IO file(s) (1 file(s) / 256MB)
queryA: Laying out IO file(s) (1 file(s) / 256MB)
queryB: Laying out IO file(s) (1 file(s) / 256MB)
bgupdater: Laying out IO file(s) (1 file(s) / 64MB)
bgwriter: (groupid=0, jobs=1): err= 0: pid=3745: Sat May 18 11:41:08 2013
write: io=41164KB, bw=701900B/s, iops=170, runt= 60054msec
slat (usec): min=11, max=412874, avg=184.80, stdev=6240.48
clat (msec): min=3, max=826, avg=186.62, stdev=115.32
lat (msec): min=3, max=826, avg=186.82, stdev=115.72
bw (KB /s): min= 0, max= 1230, per=98.55%, avg=690.82,
stdev=231.61
lat (msec) : 4=0.02%, 10=0.12%, 20=0.49%, 50=8.64%, 100=19.43%
lat (msec) : 250=39.74%, 500=30.81%, 750=1.04%, 1000=0.02%
cpu : usr=0.22%, sys=0.45%, ctx=10821, majf=0, minf=21
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.2%, 32=109.8%,
>=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%,
>=64=0.0%
issued : total=r=0/w=10260/d=0, short=r=0/w=0/d=0
queryA: (groupid=0, jobs=1): err= 0: pid=3746: Sat May 18 11:41:08 2013
read : io=17482MB, bw=298351KB/s, iops=74587, runt= 60001msec
clat (usec): min=1, max=11446, avg= 3.92, stdev=29.76
lat (usec): min=2, max=12278, avg= 5.37, stdev=34.68
bw (KB /s): min= 0, max=335136, per=54.65%, avg=295877.00,
stdev=34918.18
lat (usec) : 2=0.01%, 4=44.30%, 10=55.26%, 20=0.32%, 50=0.07%
lat (usec) : 100=0.01%, 250=0.02%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%
cpu : usr=87.11%, sys=12.06%, ctx=22949, majf=0, minf=4549229
IO depths : 1=107.7%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
issued : total=r=4475343/w=0/d=0, short=r=0/w=0/d=0
queryB: (groupid=0, jobs=1): err= 0: pid=3747: Sat May 18 11:41:08 2013
read : io=14257MB, bw=243312KB/s, iops=60828, runt= 60001msec
clat (usec): min=1, max=18287, avg= 4.18, stdev=37.17
lat (usec): min=2, max=18288, avg= 5.64, stdev=44.27
bw (KB /s): min= 0, max=270139, per=44.59%, avg=241370.27,
stdev=34771.40
lat (usec) : 2=0.01%, 4=41.55%, 10=57.84%, 20=0.33%, 50=0.15%
lat (usec) : 100=0.02%, 250=0.07%, 500=0.01%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%
cpu : usr=84.55%, sys=10.57%, ctx=38532, majf=0, minf=3774059
IO depths : 1=108.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
issued : total=r=3649742/w=0/d=0, short=r=0/w=0/d=0
bgupdater: (groupid=0, jobs=1): err= 0: pid=3748: Sat May 18 11:41:08 2013
read : io=1092.0KB, bw=18625B/s, iops=4, runt= 60036msec
slat (usec): min=9, max=104109, avg=710.32, stdev=6958.17
clat (usec): min=350, max=435402, avg=123958.98, stdev=125840.77
lat (usec): min=381, max=435421, avg=124738.53, stdev=126435.26
bw (KB /s): min= 0, max= 71, per=0.00%, avg=18.15, stdev=12.30
write: io=974848B, bw=16237B/s, iops=3, runt= 60036msec
slat (usec): min=12, max=98337, avg=1542.04, stdev=8831.57
clat (msec): min=11, max=453, avg=107.43, stdev=89.92
lat (msec): min=11, max=546, avg=109.14, stdev=95.59
bw (KB /s): min= 5, max= 43, per=2.23%, avg=15.60, stdev= 6.84
lat (usec) : 500=0.59%, 750=0.20%, 1000=0.20%
lat (msec) : 4=5.09%, 10=17.03%, 20=4.31%, 50=10.18%, 100=20.94%
lat (msec) : 250=25.05%, 500=16.44%
cpu : usr=0.02%, sys=0.04%, ctx=587, majf=0, minf=21
IO depths : 1=107.4%, 2=0.4%, 4=0.8%, 8=1.6%, 16=0.2%, 32=0.0%,
>=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
issued : total=r=273/w=238/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=31740MB, aggrb=541365KB/s, minb=18KB/s, maxb=298351KB/s,
mint=60001msec, maxt=60036msec
WRITE: io=42116KB, aggrb=701KB/s, minb=15KB/s, maxb=685KB/s,
mint=60036msec, maxt=60054msec
Disk stats (read/write):
dm-1: ios=298/13834, merge=0/0, ticks=37382/2405825, in_queue=2449686,
util=99.95%, aggrios=300/13851, aggrmerge=0/0, aggrticks=37638/2414256,
aggrin_queue=2451895, aggrutil=99.93%
dm-0: ios=300/13851, merge=0/0, ticks=37638/2414256, in_queue=2451895,
util=99.93%, aggrios=300/11782, aggrmerge=0/2069, aggrticks=37583/2145209,
aggrin_queue=2182787, aggrutil=99.93%
sda: ios=300/11782, merge=0/2069, ticks=37583/2145209, in_queue=2182787,
util=99.93%
============================================================================
bgwriter: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio,
iodepth=32
queryA: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=mmap, iodepth=1
queryB: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=mmap, iodepth=1
bgupdater: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio,
iodepth=16
fio-2.0.13
Starting 4 processes
bgwriter: (groupid=0, jobs=1): err= 0: pid=11660: Sat May 18 11:43:49 2013
write: io=21908KB, bw=372270 B/s, iops=90 , runt= 60262msec
slat (usec): min=9 , max=385032 , avg=502.70, stdev=12025.44
clat (msec): min=4 , max=1755 , avg=353.33, stdev=257.27
lat (msec): min=4 , max=1755 , avg=353.83, stdev=257.85
bw (KB/s) : min= 0, max= 780, per=74.19%, avg=369.45,
stdev=156.04
lat (msec) : 10=0.77%, 20=3.56%, 50=9.93%, 100=8.46%, 250=11.42%
lat (msec) : 500=41.28%, 750=17.15%, 1000=5.99%, 2000=2.00%
cpu : usr=0.15%, sys=0.31%, ctx=5633, majf=0, minf=21
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.3%, 32=109.5%,
>=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%,
>=64=0.0%
issued : total=r=0/w=5446/d=0, short=r=0/w=0/d=0
queryA: (groupid=0, jobs=1): err= 0: pid=11661: Sat May 18 11:43:49 2013
read : io=2232.0KB, bw=37939 B/s, iops=9 , runt= 60242msec
clat (msec): min=3 , max=881 , avg=107.95, stdev=139.78
lat (msec): min=3 , max=881 , avg=107.95, stdev=139.78
bw (KB/s) : min= 0, max= 102, per=17.30%, avg=36.84, stdev=20.55
lat (msec) : 4=0.18%, 10=3.94%, 20=35.66%, 50=21.51%, 100=7.71%
lat (msec) : 250=10.75%, 500=19.00%, 750=0.90%, 1000=0.36%
cpu : usr=0.01%, sys=0.12%, ctx=686, majf=610, minf=26
IO depths : 1=109.3%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
issued : total=r=558/w=0/d=0, short=r=0/w=0/d=0
queryB: (groupid=0, jobs=1): err= 0: pid=11662: Sat May 18 11:43:49 2013
read : io=2444.0KB, bw=41541 B/s, iops=10 , runt= 60245msec
clat (msec): min=4 , max=932 , avg=98.59, stdev=135.55
lat (msec): min=4 , max=932 , avg=98.59, stdev=135.55
bw (KB/s) : min= 0, max= 107, per=19.13%, avg=40.75, stdev=20.82
lat (msec) : 10=4.75%, 20=39.12%, 50=20.62%, 100=7.53%, 250=10.47%
lat (msec) : 500=16.37%, 750=0.98%, 1000=0.16%
cpu : usr=0.03%, sys=0.11%, ctx=690, majf=657, minf=26
IO depths : 1=107.5%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
>=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
issued : total=r=611/w=0/d=0, short=r=0/w=0/d=0
bgupdater: (groupid=0, jobs=1): err= 0: pid=11663: Sat May 18 11:43:49
2013
read : io=8180.0KB, bw=138901 B/s, iops=33 , runt= 60304msec
slat (usec): min=6 , max=181007 , avg=146.99, stdev=4496.16
clat (usec): min=80 , max=961158 , avg=178582.45, stdev=147791.89
lat (msec): min=3 , max=961 , avg=178.74, stdev=147.69
bw (KB/s) : min= 0, max= 345, per=66.90%, avg=142.51, stdev=78.70
write: io=8140.0KB, bw=138222 B/s, iops=33 , runt= 60304msec
slat (usec): min=8 , max=391440 , avg=1030.35, stdev=15704.22
clat (msec): min=4 , max=1307 , avg=293.98, stdev=215.99
lat (msec): min=4 , max=1307 , avg=295.05, stdev=217.49
bw (KB/s) : min= 6, max= 333, per=27.86%, avg=138.72, stdev=62.52
lat (usec) : 100=0.02%, 250=0.02%
lat (msec) : 4=0.07%, 10=1.43%, 20=6.72%, 50=11.34%, 100=12.32%
lat (msec) : 250=24.85%, 500=34.10%, 750=7.95%, 1000=1.35%, 2000=0.20%
cpu : usr=0.16%, sys=0.22%, ctx=4230, majf=0, minf=21
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=108.6%, 32=0.0%,
>=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%,
>=64=0.0%
issued : total=r=2036/w=2029/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=12856KB, aggrb=213KB/s, minb=37KB/s, maxb=135KB/s,
mint=60242msec, maxt=60304msec
WRITE: io=30048KB, aggrb=498KB/s, minb=134KB/s, maxb=363KB/s,
mint=60262msec, maxt=60304msec
Disk stats (read/write):
dm-1: ios=3495/9099, merge=0/0, ticks=517051/2958255,
in_queue=3476437, util=99.97%, aggrios=3495/9115, aggrmerge=0/0,
aggrticks=518008/2959075, aggrin_queue=3477084, aggrutil=99.95%
dm-0: ios=3495/9115, merge=0/0, ticks=518008/2959075,
in_queue=3477084, util=99.95%, aggrios=3493/8375, aggrmerge=2/740,
aggrticks=516946/2742061, aggrin_queue=3259004, aggrutil=99.95%
sda: ios=3493/8375, merge=2/740, ticks=516946/2742061, in_queue=3259004,
util=99.95%
From: Jens Axboe <axboe@xxxxxxxxx>
To: Edoardo Comar/UK/IBM@IBMGB,
Cc: fio@xxxxxxxxxxxxxxx
Date: 18/05/2013 08:01
Subject: Re: output - aggregate read bandwitdh
Sent by: fio-owner@xxxxxxxxxxxxxxx
On Sat, May 18 2013, Jens Axboe wrote:
> On Sat, May 18 2013, Edoardo Comar wrote:
> > I have a problem with fio built today from GIT
> > running the jobfile below, the aggregate bandwidth for 'read' output
by
> > the latest fio is excessive
> > while it seems reasonable with an older fio version.
> >
> > fio-2.0.13 READ: io=51500KB, aggrb=857KB/s,
> >
> > fio-2.1-6-g4d01 READ: io=31057MB, aggrb=530017KB/s,
> >
> >
> > You should easily reproduce this with the data below.
> > I have reproduced it on my laptop (RHEL 6.4) as well as on other RHEL
> > systems.
> > on my laptop
> > READ: io=13028KB, aggrb=216KB/s,
> > or
> > READ: io=33204MB, aggrb=566519KB/s,
> >
> > running the same configuration
> >
> > I believe this to be a bug.
>
> I wonder if this is a regression from 2.0.15 to 2.1. The gfio stuff got
> merged in along the way, could definitely be an accounting issue
> introduced there. I'll take a look.
Doesn't reproduce here for me. Could you try a make clean, the configure
and make again? Just to be on the safe side...
--
Jens Axboe
--
To unsubscribe from this list: send the line "unsubscribe fio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
--
To unsubscribe from this list: send the line "unsubscribe fio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
