Re: Increase queue_depth in KVM

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello,

Steffen, Thanks for Your reply. Sorry but I was on holidays, now I'm back and still digging into my problem.. :(


I've read thousands of google links but can't find anything which could help me.

- tried all qemu drive IO(io=) and cache(cache=) modes, nothing could come even close to the results I'm getting outside KVM.
- enabling blk-mq inside KVM guest didn't help
- enabling iothreads didn't help
- 'queues' parameter in my libvirt schemas can be only applied to 'virtio-serial', can't use it in virtio-scsi nor virtio-blk)
- I've seen some people using 'num_queues' but I don't have this parameter in my schemas(libvirt version = 1.3.1, qemu version = 2.5.0)



So, is there really no way to increase queue_depth to rbd device in kvm domain or any other way to achieve results similar to those obtained outside KVM? :/

wt., 26 cze 2018 o 15:19 Steffen Winther Sørensen <stefws@xxxxxxxxx> napisał(a):


> On 26 Jun 2018, at 14.04, Damian Dabrowski <scooty96@xxxxxxxxx> wrote:
>
> Hi Stefan, thanks for reply.
>
> Unfortunately it didn't work.
>
> disk config:
>     <disk type='network' device='disk'>
>       <driver name='qemu' type='raw' cache='directsync' io='native' discard='unmap'/>
>       <auth username='compute'>
>         <secret type='ceph' uuid='a5d0dd94-57c4-ae55-ffe0-7e3732a24455'/>
>       </auth>
>       <source protocol='rbd' name='volumes-nvme/volume-ce247187-a625-49f1-bacd-fc03df215395'>
>         <host name='172.31.0.1' port='6789'/>
>         <host name='172.31.0.2' port='6789'/>
>         <host name='172.31.0.3' port='6789'/>
>       </source>
>       <target dev='sdb' bus='scsi'/>
>       <serial>ce247187-a625-49f1-bacd-fc03df215395</serial>
>       <address type='drive' controller='0' bus='0' target='0' unit='2'/>
>     </disk>
>
>
> Controller config:
>     <controller type='scsi' index='0' model='virtio-scsi'>
>       <driver queues='8'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
>     </controller>
>
>
> benchmark command: fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randwrite --time_based --runtime=60 --write_iops_log=write_results --numjobs=8
>
> And I'm still getting very low random write IOPS inside KVM instance with 8vcores(3-5k compared to 20k+ outside KVM)
>
> Maybe do You have any idea how to deal with it?
What about trying with io=‘threads’ and/or maybe cache=’none’ or swap from virtio-scsi to blk-mq?

Other people had similar issues, try asking ‘G’
https://serverfault.com/questions/425607/kvm-guest-io-is-much-slower-than-host-io-is-that-normal
https://wiki.mikejung.biz/KVM_/_Xen

/Steffen

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux