On Fri, Apr 24, 2020 at 03:25:40PM +0900, Chirantan Ekbote wrote: > Use simple round-robin scheduling based on the `unique` field of the > fuse request to spread requests across multiple queues, if supported by > the device. Multiqueue is not intended to be used this way and this patch will reduce performance*. I don't think it should be merged. * I know it increases performance for you :) but hear me out: The purpose of multiqueue is for SMP scalability. It allows queues to be processed with CPU/NUMA affinity to the vCPU that submitted the request (i.e. the virtqueue processing thread runs on a sibling physical CPU core). Each queue has its own MSI-X interrupt so that completion interrupts can be processed on the same vCPU that submitted the request. Spreading requests across queues defeats all this. Virtqueue processing threads that are located in the wrong place will now process the requests. Completion interrupts will wake up a vCPU that did not submit the request and IPIs are necessary to notify the vCPU that originally submitted the request. Even if you don't care about SMP performance, using multiqueue as a workaround for missing request parallelism still won't yield the best results. The guest should be able to submit up to the maximum queue depth of the physical storage device. Many Linux block drivers have max queue depths of 64. This would require 64 virtqueues (plus the queue selection algorithm would have to utilize each one) and shows how wasteful this approach is. Instead of modifying the guest driver, please implement request parallelism in your device implementation. Device implementations should pop a virtqueue element, submit I/O, and then move on to the next virtqueue element instead of waiting for the I/O to complete. This can be done with a thread pool, coroutines, async I/O APIs, etc. The C implementation of virtiofsd has request parallelism and there is work underway to do it in Rust for cloud-hypervisor too. Perhaps you can try the cloud-hypervisor code, I have CCed Sergio Lopez? Thanks, Stefan
Attachment:
signature.asc
Description: PGP signature
