IO Depth lower than requested

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I'm testing some NVMe drives and I notice the IO Depth reported by fio
output is lower than what I specify in the job.  Is the command-line
output showing the attempted depth or the actual achieved depth?
Here's an example output where I tried iodepth=32.  The first line (IO
depths) shows a mix of 8 and 16.  The next 2 lines (submit and
complete) show 4.  What's the difference between the first and second
lines?  I was expecting "submit" to be 32 and "complete" to maybe be
lower based on hardware and OS limitations.  Oddly enough, if I
specify iodepth=4 then performance is actually a bit better (32K
instead of 35K) even though the submit/complete depths shown remain
the same.
Also — is there a way to find out what is limiting this?  The Micron
9300 Pro should support QD=512 according to its spec sheet.

% fio --randrepeat=1 --name=test --bs=4k --iodepth=32 --size=1G
--readwrite=randread --directory /Volumes/Micron9300/elliott
--ioengine posixaio
test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
4096B-4096B, ioengine=posixaio, iodepth=32
fio-3.30
Starting 1 process
Jobs: 1 (f=1): [r(1)][100.0%][r=137MiB/s][r=35.1k IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=6903: Fri Apr 22 12:25:34 2022
  read: IOPS=35.0k, BW=137MiB/s (143MB/s)(1024MiB/7498msec)
    slat (nsec): min=0, max=46000, avg=823.84, stdev=784.94
    clat (usec): min=179, max=874, avg=455.01, stdev=50.98
     lat (usec): min=180, max=875, avg=455.84, stdev=50.88
    clat percentiles (usec):
     |  1.00th=[  318],  5.00th=[  367], 10.00th=[  392], 20.00th=[  416],
     | 30.00th=[  433], 40.00th=[  445], 50.00th=[  457], 60.00th=[  469],
     | 70.00th=[  482], 80.00th=[  494], 90.00th=[  515], 95.00th=[  537],
     | 99.00th=[  578], 99.50th=[  594], 99.90th=[  652], 99.95th=[  668],
     | 99.99th=[  717]
   bw (  KiB/s): min=139419, max=140595, per=100.00%, avg=140110.71,
stdev=340.75, samples=14
   iops        : min=34854, max=35148, avg=35027.43, stdev=85.14, samples=14
  lat (usec)   : 250=0.06%, 500=82.67%, 750=17.27%, 1000=0.01%
  cpu          : usr=19.97%, sys=13.74%, ctx=195959, majf=0, minf=28
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=43.4%, 16=56.6%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=99.1%, 8=0.9%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=262144,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: bw=137MiB/s (143MB/s), 137MiB/s-137MiB/s (143MB/s-143MB/s),
io=1024MiB (1074MB), run=7498-7498msec

Thanks,
Elliott




[Index of Archives]     [Linux Kernel]     [Linux SCSI]     [Linux IDE]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux SCSI]

  Powered by Linux