Hi,
I test in VM with fio, here is the config:
[global]
direct=1
ioengine=aio
iodepth=1
[sequence read 4K]
rw=read
bs=4K
size=1024m
directory=/mnt
filename=test
sequence read 4K: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=1
fio-2.1.3
Starting 1 process
sequence read 4K: Laying out IO file(s) (1 file(s) / 1024MB)
^Cbs: 1 (f=1): [R] [18.0% done] [1994KB/0KB/0KB /s] [498/0/0 iops] [eta 07m:14s]
fio: terminating on signal 2
sequence read 4K: (groupid=0, jobs=1): err= 0: pid=1156: Fri Nov 21 12:32:53 2014
read : io=187408KB, bw=1984.1KB/s, iops=496, runt= 94417msec
slat (usec): min=22, max=878, avg=48.36, stdev=22.63
clat (usec): min=1335, max=17618, avg=1956.45, stdev=247.26
lat (usec): min=1371, max=17680, avg=2006.97, stdev=248.47
clat percentiles (usec):
| 1.00th=[ 1560], 5.00th=[ 1640], 10.00th=[ 1704], 20.00th=[ 1784],
| 30.00th=[ 1848], 40.00th=[ 1896], 50.00th=[ 1944], 60.00th=[ 1992],
| 70.00th=[ 2064], 80.00th=[ 2128], 90.00th=[ 2192], 95.00th=[ 2288],
| 99.00th=[ 2448], 99.50th=[ 2640], 99.90th=[ 3856], 99.95th=[ 4256],
| 99.99th=[ 9408]
bw (KB /s): min= 1772, max= 2248, per=100.00%, avg=1986.55, stdev=85.76
lat (msec) : 2=60.69%, 4=39.23%, 10=0.07%, 20=0.01%
cpu : usr=1.92%, sys=2.98%, ctx=47125, majf=0, minf=28
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=46852/w=0/d=0, short=r=0/w=0/d=0
Run status group 0 (all jobs):
READ: io=187408KB, aggrb=1984KB/s, minb=1984KB/s, maxb=1984KB/s, mint=94417msec, maxt=94417msec
Disk stats (read/write):
sda: ios=46754/11, merge=0/10, ticks=91144/40, in_queue=91124, util=96.73%
the
the rados benchmark:
# rados -p volumes bench 60 seq -b 4096 -t 1
Total time run: 44.922178
Total reads made: 24507
Read size: 4096
Bandwidth (MB/sec): 2.131
Average Latency: 0.00183069
Max latency: 0.004598
Min latency: 0.001224
|
Hi,
I don't have tested yet rbd readhead,
but maybe do you reach qemu limit. (by default qemu can use only 1thread/1core to manage ios, check you qemu cpu).
Do you have some performance results ? how many iops ?
but I have had 4x improvement in qemu-kvm, with virtio-scsi + num_queues + lasts kernel.
(4k seq coalesced reads in qemu, was doing bigger iops to ceph).
libvirt : <controller type='scsi' index='0' model='virtio-scsi' num_queues='8'/>
Regards,
Alexandre
----- Mail original -----
De: "duan xufeng" <duan.xufeng@xxxxxxxxxx>
À: "ceph-users" <ceph-users@xxxxxxxx>
Cc: "si dawei" <si.dawei@xxxxxxxxxx>
Envoyé: Vendredi 21 Novembre 2014 03:58:38
Objet: [ceph-users] RBD read-ahead didn't improve 4K read performance
hi，
I upgraded CEPH to 0.87 for rbd readahead , but can't see any performance improvement in 4K seq read in the VM.
How can I know if the readahead is take effect?
thanks.
ceph.conf
[client]
rbd_cache = true
rbd_cache_size = 335544320
rbd_cache_max_dirty = 251658240
rbd_cache_target_dirty = 167772160
rbd readahead trigger requests = 1
rbd readahead max bytes = 4194304
rbd readahead disable after bytes = 0
--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s). If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited. If you have received this mail in error, please delete it and notify us immediately.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
-------------------------------------------------------- ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s). If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited. If you have received this mail in error, please delete it and notify us immediately.
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
