Hello,
Steffen, Thanks for Your reply. Sorry but I was on holidays, now I'm back and still digging into my problem.. :(
I've read thousands of google links but can't find anything which could help me.
- tried all qemu drive IO(io=) and cache(cache=) modes, nothing could come even close to the results I'm getting outside KVM.
- enabling blk-mq inside KVM guest didn't help
- enabling iothreads didn't help
- 'queues' parameter in my libvirt schemas can be only applied to 'virtio-serial', can't use it in virtio-scsi nor virtio-blk)
- I've seen some people using 'num_queues' but I don't have this parameter in my schemas(libvirt version = 1.3.1, qemu version = 2.5.0)
So, is there really no way to increase queue_depth to rbd device in kvm domain or any other way to achieve results similar to those obtained outside KVM? :/
wt., 26 cze 2018 o 15:19 Steffen Winther Sørensen <stefws@xxxxxxxxx> napisał(a):
> On 26 Jun 2018, at 14.04, Damian Dabrowski <scooty96@xxxxxxxxx> wrote:
>
> Hi Stefan, thanks for reply.
>
> Unfortunately it didn't work.
>
> disk config:
> <disk type='network' device='disk'>
> <driver name='qemu' type='raw' cache='directsync' io='native' discard='unmap'/>
> <auth username='compute'>
> <secret type='ceph' uuid='a5d0dd94-57c4-ae55-ffe0-7e3732a24455'/>
> </auth>
> <source protocol='rbd' name='volumes-nvme/volume-ce247187-a625-49f1-bacd-fc03df215395'>
> <host name='172.31.0.1' port='6789'/>
> <host name='172.31.0.2' port='6789'/>
> <host name='172.31.0.3' port='6789'/>
> </source>
> <target dev='sdb' bus='scsi'/>
> <serial>ce247187-a625-49f1-bacd-fc03df215395</serial>
> <address type='drive' controller='0' bus='0' target='0' unit='2'/>
> </disk>
>
>
> Controller config:
> <controller type='scsi' index='0' model='virtio-scsi'>
> <driver queues='8'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
> </controller>
>
>
> benchmark command: fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randwrite --time_based --runtime=60 --write_iops_log=write_results --numjobs=8
>
> And I'm still getting very low random write IOPS inside KVM instance with 8vcores(3-5k compared to 20k+ outside KVM)
>
> Maybe do You have any idea how to deal with it?
What about trying with io=‘threads’ and/or maybe cache=’none’ or swap from virtio-scsi to blk-mq?
Other people had similar issues, try asking ‘G’
https://serverfault.com/questions/425607/kvm-guest-io-is-much-slower-than-host-io-is-that-normal
https://wiki.mikejung.biz/KVM_/_Xen
/Steffen
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
