Hi, I run randow fio with rwmixread=70, and found read iops is 707, write is 303. (reference the following). This value is less than random write and read value. The 4K random write IOPs is 529 and 4k randread IOPs is 11343. Apart from rw type is different, other parameters are all same.
I do not understand why mix write and read will have so huge impact on performance. All random IOs. thanks.fio -filename=/dev/rbd2 -direct=1 -iodepth 64 -thread -rw=randrw -rwmixread=70 -ioengine=libaio -bs=4k -size=100G -numjobs=1 -runtime=1000 -group_reporting -name=mytest1
mytest1: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.8
time 7423 cycles_start=1062103697308843
Starting 1 thread
Jobs: 1 (f=1): [m(1)] [100.0% done] [2144KB/760KB/0KB /s] [536/190/0 iops] [eta 00m:00s]
mytest1: (groupid=0, jobs=1): err= 0: pid=7425: Sat Apr 30 08:55:14 2016
read : io=2765.2MB, bw=2830.5KB/s, iops=707, runt=1000393msec
slat (usec): min=2, max=268, avg= 8.93, stdev= 4.17
clat (usec): min=203, max=1939.9K, avg=34039.43, stdev=93674.48
lat (usec): min=207, max=1939.9K, avg=34048.93, stdev=93674.50
clat percentiles (usec):
| 1.00th=[ 516], 5.00th=[ 836], 10.00th=[ 1112], 20.00th=[ 1448],
| 30.00th=[ 1736], 40.00th=[ 6944], 50.00th=[13376], 60.00th=[17280],
| 70.00th=[21888], 80.00th=[30848], 90.00th=[49920], 95.00th=[103936],
| 99.00th=[552960], 99.50th=[675840], 99.90th=[880640], 99.95th=[954368],
| 99.99th=[1105920]
bw (KB /s): min= 350, max= 5944, per=100.00%, avg=2837.77, stdev=1272.84
write: io=1184.8MB, bw=1212.8KB/s, iops=303, runt=1000393msec
slat (usec): min=2, max=310, avg= 9.35, stdev= 4.50
clat (msec): min=5, max=2210, avg=131.60, stdev=226.47
lat (msec): min=5, max=2210, avg=131.61, stdev=226.47
clat percentiles (msec):
| 1.00th=[ 9], 5.00th=[ 13], 10.00th=[ 15], 20.00th=[ 20],
| 30.00th=[ 25], 40.00th=[ 34], 50.00th=[ 44], 60.00th=[ 61],
| 70.00th=[ 84], 80.00th=[ 125], 90.00th=[ 449], 95.00th=[ 709],
| 99.00th=[ 1037], 99.50th=[ 1139], 99.90th=[ 1369], 99.95th=[ 1450],
| 99.99th=[ 1663]
bw (KB /s): min= 40, max= 2562, per=100.00%, avg=1215.62, stdev=564.19
lat (usec) : 250=0.01%, 500=0.60%, 750=1.94%, 1000=2.95%
lat (msec) : 2=18.69%, 4=2.46%, 10=4.21%, 20=22.05%, 50=26.40%
lat (msec) : 100=9.65%, 250=4.64%, 500=2.76%, 750=2.13%, 1000=1.11%
lat (msec) : 2000=0.39%, >=2000=0.01%
cpu : usr=0.83%, sys=1.47%, ctx=971080, majf=0, minf=1
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued : total=r=707885/w=303294/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: io=2765.2MB, aggrb=2830KB/s, minb=2830KB/s, maxb=2830KB/s, mint=1000393msec, maxt=1000393msec
WRITE: io=1184.8MB, aggrb=1212KB/s, minb=1212KB/s, maxb=1212KB/s, mint=1000393msec, maxt=1000393msec
Disk stats (read/write):
rbd2: ios=707885/303293, merge=0/0, ticks=24085792/39904840, in_queue=64045864, util=100.00%
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
