You may be on to something!
I tried 3 drives, got the exact same results. See attached.
[global]
ioengine=windowsaio
blocksize=64k
direct=1
thread
size=250m
time_based
runtime=10
[asdf]
filename=F\:\\testfile:G\:\\testfile:H\:\\testfile
;:I\:\\testfile:J\:\\testfile:K\:\\testfile:L\:\\testfile:M\:\\testfile:N\:\\testfile
;:O\:\\testfile:P\:\\testfile:Q\:\\testfile
-Dave
-----Original Message-----
From: Sitsofe Wheeler [mailto:sitsofe@xxxxxxxxx]
Sent: Tuesday, October 31, 2017 3:15 PM
To: David Hare <david.hare@xxxxxxxxxxxxxxx>
Cc: Jens Axboe <axboe@xxxxxxxxx>; fio@xxxxxxxxxxxxxxx
Subject: Re: FIO windows
One idea is that you are seeing the effect of trying to do I/O to a file that is not a multiple of the blocksize. In theory if you have size=1g and you have 9 files then each file ends up being 1024**3/9.0 ~ 119304647.1111111 big (see http://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-filename
for where this is described). Could it be that Windows goes on to make a file that is smaller than what we were asking for?
If this theory were right you might see a similar problem if you were only using 3 files.
On 31 October 2017 at 22:06, David Hare <david.hare@xxxxxxxxxxxxxxx> wrote:
> Yes.. I made a typo when I changed it back, sorry.
>
> -----Original Message-----
> From: Sitsofe Wheeler [mailto:sitsofe@xxxxxxxxx]
> Sent: Tuesday, October 31, 2017 3:05 PM
> To: David Hare <david.hare@xxxxxxxxxxxxxxx>
> Cc: Jens Axboe <axboe@xxxxxxxxx>; fio@xxxxxxxxxxxxxxx
> Subject: Re: FIO windows
>
> Yes that's right. Also previously did you mean you had set size=512m
> even though you wrote size=512g ?
>
> On 31 October 2017 at 22:03, David Hare <david.hare@xxxxxxxxxxxxxxx> wrote:
>> I assume you want me to change the size parameter with a 64k
>> blocksize as everything is working with 16k blocksize?
>>
>> -----Original Message-----
>> From: Sitsofe Wheeler [mailto:sitsofe@xxxxxxxxx]
>> Sent: Tuesday, October 31, 2017 2:54 PM
>> To: David Hare <david.hare@xxxxxxxxxxxxxxx>
>> Cc: Jens Axboe <axboe@xxxxxxxxx>; fio@xxxxxxxxxxxxxxx
>> Subject: Re: FIO windows
>>
>> Hi,
>>
>> Can you add unlink=1 and keep reducing the size parameter (e.g. down
>> to 128m then down to 16m then down to 4m then down to 1m then down to
>> 512k etc)?
>>
>> Can you attach the full output that's produced it fails with this
>> reduced job?
>>
>> IF you are make the problem happen with very little I/O being done
>> (i.e. the job bombs out after doing less than 1MiBytes worth of I/O)
>> you can try adding --debug=all to the job and seeing if that offers
>> any clues as to what the last thing it was doing was?
>>
>> On 31 October 2017 at 21:46, David Hare <david.hare@xxxxxxxxxxxxxxx>
>> wrote:
>>> It was ok with or without the colon, the size didn’t seem to make a
>>> difference, but blocksize did.. see the commented block sizes below.
>>>
>>> fio2.fio
>>> [global]
>>>
>>> ioengine=windowsaio
>>>
>>> ;blocksize=64k - error
>>> ;blocksize=32k - error
>>> ;blocksize=16k - no error
>>>
>>> blocksize=16k
>>>
>>> direct=1
>>>
>>> thread
>>>
>>> size=512g
>>>
>>>
>>>
>>> time_based
>>> runtime=10
>>>
>>> [asdf]
>>> filename=F\:\\testfile:G\:\\testfile:H\:\\testfile:I\:\\testfile:J\:
>>> \ \ testfile:K\:\\testfile:L\:\\testfile:M\:\\testfile:P\:\\testfile
>>>
>>> Results:
>>> Run status group 0 (all jobs):
>>> READ: bw=141MiB/s (148MB/s), 141MiB/s-141MiB/s (148MB/s-148MB/s),
>>> io=1413MiB (1481MB), run=10001-10001msec
>>>
>>>
>>> -Dave
>
> --
> Sitsofe | http://sucs.org/~sits/
>
>
> Disclaimer
>
> The information contained in this communication from the sender is
> confidential. It is intended solely for use by the recipient and
> others authorized to receive it. If you are not the recipient, you are
> hereby notified that any disclosure, copying, distribution or taking
> action in relation of the contents of this information is strictly
> prohibited and may be unlawful.
--
Sitsofe | http://sucs.org/~sits/
asdf: (g=0): rw=read, bs=(R) 64.0KiB-64.0KiB, (W) 64.0KiB-64.0KiB, (T) 64.0KiB-64.0KiB, ioengine=windowsaio, iodepth=1
fio-3.1
Starting 1 thread
fio: pid=7820, err=22/file:ioengines.c:333, func=td_io_queue, error=Invalid argument
asdf: (groupid=0, jobs=1): err=22 (file:ioengines.c:333, func=td_io_queue, error=Invalid argument): pid=7820: Tue Oct 31 22:31:43 2017
read: IOPS=6010, BW=376MiB/s (394MB/s)(250MiB/666msec)
slat (usec): min=21, max=2127, avg=24.25, stdev=56.38
clat (usec): min=114, max=1974, avg=140.16, stdev=36.96
lat (usec): min=136, max=2349, avg=164.41, stdev=69.66
clat percentiles (usec):
| 1.00th=[ 116], 5.00th=[ 117], 10.00th=[ 118], 20.00th=[ 120],
| 30.00th=[ 122], 40.00th=[ 124], 50.00th=[ 128], 60.00th=[ 155],
| 70.00th=[ 157], 80.00th=[ 159], 90.00th=[ 161], 95.00th=[ 163],
| 99.00th=[ 176], 99.50th=[ 212], 99.90th=[ 396], 99.95th=[ 424],
| 99.99th=[ 1975]
bw ( KiB/s): min=383363, max=383363, per=99.68%, avg=383363.00, stdev= 0.00, samples=1
iops : min= 5990, max= 5990, avg=5990.00, stdev= 0.00, samples=1
lat (usec) : 250=99.68%, 500=0.27%
lat (msec) : 2=0.02%
cpu : usr=0.00%, sys=0.00%, ctx=0, majf=0, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.1%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=4003,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=376MiB/s (394MB/s), 376MiB/s-376MiB/s (394MB/s-394MB/s), io=250MiB (262MB), run=666-666msec
asdf: (g=0): rw=read, bs=(R) 64.0KiB-64.0KiB, (W) 64.0KiB-64.0KiB, (T) 64.0KiB-64.0KiB, ioengine=windowsaio, iodepth=1
fio-3.1
Starting 1 thread
asdf: (groupid=0, jobs=1): err= 0: pid=7680: Tue Oct 31 22:31:35 2017
read: IOPS=5771, BW=361MiB/s (378MB/s)(3608MiB/10001msec)
slat (usec): min=21, max=2141, avg=23.22, stdev=18.24
clat (usec): min=116, max=2023, avg=147.97, stdev=36.77
lat (usec): min=138, max=2401, avg=171.20, stdev=41.32
clat percentiles (usec):
| 1.00th=[ 120], 5.00th=[ 121], 10.00th=[ 122], 20.00th=[ 125],
| 30.00th=[ 128], 40.00th=[ 155], 50.00th=[ 155], 60.00th=[ 157],
| 70.00th=[ 157], 80.00th=[ 159], 90.00th=[ 161], 95.00th=[ 161],
| 99.00th=[ 180], 99.50th=[ 219], 99.90th=[ 371], 99.95th=[ 383],
| 99.99th=[ 1942]
bw ( KiB/s): min=354375, max=374280, per=98.72%, avg=364676.74, stdev=5516.54, samples=19
iops : min= 5537, max= 5848, avg=5697.63, stdev=86.20, samples=19
lat (usec) : 250=99.70%, 500=0.27%
lat (msec) : 2=0.03%, 4=0.01%
cpu : usr=0.00%, sys=10.00%, ctx=0, majf=0, minf=0
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=57723,0,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=361MiB/s (378MB/s), 361MiB/s-361MiB/s (378MB/s-378MB/s), io=3608MiB (3783MB), run=10001-10001msec
