Hi:
| environment:
node A :client with fio running |
|
node B :brick
network：1G
Problem is fio with libgfapi getting poor performance than fio whit fuse,what could be the reason?
[fuse]
/usr/local/bin/fio --filename=/home/gluster/file --direct=1 --rw=randwrite --bs=32k --size=20G --numjobs=60 --runtime=120 --group_reporting --name=test-read --ioengine=libaio --iodepth=1
test-read: (g=0): rw=randwrite, bs=32K-32K/32K-32K/32K-32K, ioengine=libaio, iodepth=1
...
fio-2.2.5
Starting 60 processes
Jobs: 60 (f=60): [w(60)] [100.0% done] [0KB/103.8MB/0KB /s] [0/3318/0 iops] [eta 00m:00s]
test-read: (groupid=0, jobs=60): err= 0: pid=46659: Mon Jan 26 04:40:28 2015
write: io=12804MB, bw=109247KB/s, iops=3413, runt=120016msec
slat (usec): min=46, max=14914, avg=546.08, stdev=1596.17
clat (usec): min=73, max=123085, avg=17017.70, stdev=4057.91
lat (usec): min=208, max=123264, avg=17564.57, stdev=3443.93
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 11], 10.00th=[ 16], 20.00th=[ 17],
| 30.00th=[ 17], 40.00th=[ 17], 50.00th=[ 18], 60.00th=[ 18],
| 70.00th=[ 18], 80.00th=[ 19], 90.00th=[ 19], 95.00th=[ 20],
| 99.00th=[ 35], 99.50th=[ 38], 99.90th=[ 48], 99.95th=[ 56],
| 99.99th=[ 118]
bw (KB /s): min= 1418, max= 2091, per=1.67%, avg=1822.44, stdev=66.16
lat (usec) : 100=0.18%, 250=0.11%, 500=0.04%, 750=0.03%, 1000=0.04%
lat (msec) : 2=0.20%, 4=0.55%, 10=3.41%, 20=92.28%, 50=3.08%
lat (msec) : 100=0.05%, 250=0.03%
cpu : usr=0.11%, sys=0.22%, ctx=955257, majf=0, minf=1696
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=409732/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=12804MB, aggrb=109247KB/s, minb=109247KB/s, maxb=109247KB/s, mint=120016msec, maxt=120016msec
[libgfapi]
/usr/local/bin/fio --direct=1 --rw=randwrite --bs=32k --size=20g --runtime=120 --numjobs=60 --group_reporting --ioengine=gfapi --volume=gluster_*** --brick=*** --name=/file
/file: (g=0): rw=randwrite, bs=32K-32K/32K-32K/32K-32K, ioengine=gfapi, iodepth=1
...
fio-2.2.5
Starting 60 processes
Jobs: 42 (f=0): [w(3),_(5),w(1),_(1),w(1),_(1),w(1),_(1),w(1),_(2),w(4),_(3),w(1),_(1),w(1),_(1),w(1),_(1),w(3),_(1),w(5),_(1),w(20)] [5.4% done] [0KB/3168KB/0KB /s] [0/99/0 iops] [eta 36m:17s]
/file: (groupid=0, jobs=60): err= 0: pid=46325: Mon Jan 26 04:37:13 2015
write: io=11288MB, bw=96260KB/s, iops=3008, runt=120075msec
clat (usec): min=11, max=488899, avg=19927.46, stdev=15351.15
lat (usec): min=12, max=488901, avg=19928.92, stdev=15351.30
clat percentiles (usec):
| 1.00th=[ 217], 5.00th=[ 1496], 10.00th=[ 2896], 20.00th=[ 6048],
| 30.00th=[10048], 40.00th=[13888], 50.00th=[17792], 60.00th=[21888],
| 70.00th=[27008], 80.00th=[31360], 90.00th=[36096], 95.00th=[46336],
| 99.00th=[66048], 99.50th=[74240], 99.90th=[101888], 99.95th=[126464],
| 99.99th=[189440]
bw (KB /s): min= 101, max=27130, per=1.68%, avg=1615.36, stdev=900.74
lat (usec) : 20=0.34%, 50=0.19%, 100=0.14%, 250=0.44%, 500=0.65%
lat (usec) : 750=0.77%, 1000=0.87%
lat (msec) : 2=3.49%, 4=6.76%, 10=16.36%, 20=25.76%, 50=40.16%
lat (msec) : 100=3.95%, 250=0.12%, 500=0.01%
cpu : usr=0.39%, sys=0.17%, ctx=362268, majf=0, minf=3608
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=361200/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=11288MB, aggrb=96259KB/s, minb=96259KB/s, maxb=96259KB/s, mint=120075msec, maxt=120075msec
-------------------------------------------------------- ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s). If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited. If you have received this mail in error, please delete it and notify us immediately.
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-users
