Hi Stefan, thanks for reply.
Unfortunately it didn't work.
disk config:
<disk type='network' device='disk'>
<driver name='qemu' type='raw' cache='directsync' io='native' discard='unmap'/>
<auth username='compute'>
<secret type='ceph' uuid='a5d0dd94-57c4-ae55-ffe0-7e3732a24455'/>
</auth>
<source protocol='rbd' name='volumes-nvme/volume-ce247187-a625-49f1-bacd-fc03df215395'>
<host name='172.31.0.1' port='6789'/>
<host name='172.31.0.2' port='6789'/>
<host name='172.31.0.3' port='6789'/>
</source>
<target dev='sdb' bus='scsi'/>
<serial>ce247187-a625-49f1-bacd-fc03df215395</serial>
<address type='drive' controller='0' bus='0' target='0' unit='2'/>
</disk>
Controller config:
<controller type='scsi' index='0' model='virtio-scsi'>
<driver queues='8'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</controller>
benchmark command: fio --randrepeat=1 --ioengine=libaio --direct=1 --name=test --filename=test --bs=4k --iodepth=64 --size=1G --readwrite=randwrite --time_based --runtime=60 --write_iops_log=write_results --numjobs=8
And I'm still getting very low random write IOPS inside KVM instance with 8vcores(3-5k compared to 20k+ outside KVM)
Maybe do You have any idea how to deal with it?
wt., 26 cze 2018 o 09:37 Stefan Kooman <stefan@xxxxxx> napisał(a):
Quoting Damian Dabrowski (scooty96@xxxxxxxxx):
> Hello,
>
> When I mount rbd image with -o queue_depth=1024 I can see much improvement,
> generally on writes(random write improvement from 3k IOPS on standard
> queue_depth to 24k IOPS on queue_depth=1024).
>
> But is there any way to attach rbd disk to KVM instance with custom
> queue_depth? I can't find any information about it.
Not specifically "queue depth", but if you use virtio-scsi, and have a
VM with more than 1 vCPU, you can give each vCPU it's own queue [1]:
<controller type='scsi' index='0' model='virtio-scsi'>
<driver queues='N' />
</controller>
Gr. Stefan
[1]: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_tuning_and_optimization_guide/sect-virtualization_tuning_optimization_guide-blockio-techniques
--
| BIT BV http://www.bit.nl/ Kamer van Koophandel 09090351
| GPG: 0xD14839C6 +31 318 648 688 / info@xxxxxx
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
