Jason Wang <jasowang@xxxxxxxxxx> wrote on 27/11/2013 04:49:20 AM: > > On 11/24/2013 05:22 PM, Razya Ladelsky wrote: > > Hi all, > > > > I am Razya Ladelsky, I work at IBM Haifa virtualization team, which > > developed Elvis, presented by Abel Gordon at the last KVM forum: > > ELVIS video: https://www.youtube.com/watch?v=9EyweibHfEs > > ELVIS slides: https://drive.google.com/file/d/0BzyAwvVlQckeQmpnOHM5SnB5UVE > > > > > > According to the discussions that took place at the forum, upstreaming > > some of the Elvis approaches seems to be a good idea, which we would like > > to pursue. > > > > Our plan for the first patches is the following: > > > > 1.Shared vhost thread between mutiple devices > > This patch creates a worker thread and worker queue shared across multiple > > virtio devices > > We would like to modify the patch posted in > > https://github.com/abelg/virtual_io_acceleration/commit/ > 3dc6a3ce7bcbe87363c2df8a6b6fee0c14615766 > > to limit a vhost thread to serve multiple devices only if they belong to > > the same VM as Paolo suggested to avoid isolation or cgroups concerns. > > > > Another modification is related to the creation and removal of vhost > > threads, which will be discussed next. > > > > 2. Sysfs mechanism to add and remove vhost threads > > This patch allows us to add and remove vhost threads dynamically. > > > > A simpler way to control the creation of vhost threads is statically > > determining the maximum number of virtio devices per worker via a kernel > > module parameter (which is the way the previously mentioned patch is > > currently implemented) > > Any chance we can re-use the cwmq instead of inventing another > mechanism? Looks like there're lots of function duplication here. Bandan > has an RFC to do this. Thanks for the suggestion. We should certainly take a look at Bandan's patches which I guess are: http://www.mail-archive.com/kvm@xxxxxxxxxxxxxxx/msg96603.html My only concern here is that we may not be able to easily implement our polling mechanism and heuristics with cwmq. > > > > I'd like to ask for advice here about the more preferable way to go: > > Although having the sysfs mechanism provides more flexibility, it may be a > > good idea to start with a simple static parameter, and have the first > > patches as simple as possible. What do you think? > > > > 3.Add virtqueue polling mode to vhost > > Have the vhost thread poll the virtqueues with high I/O rate for new > > buffers , and avoid asking the guest to kick us. > > https://github.com/abelg/virtual_io_acceleration/commit/ > 26616133fafb7855cc80fac070b0572fd1aaf5d0 > > Maybe we can make poll_stop_idle adaptive which may help the light load > case. Consider guest is often slow than vhost, if we just have one or > two vms, polling too much may waste cpu in this case. Yes, make polling adaptive based on the amount of wasted cycles (cycles we did polling but didn't find new work) and I/O rate is a very good idea. Note we already measure and expose these values but we do not use them to adapt the polling mechanism. Having said that, note that adaptive polling may be a bit tricky. Remember that the cycles we waste polling in the vhost thread actually improves the performance of the vcpu threads because the guest is no longer require to kick (pio==exit) the host when vhost does polling. So even if we waste cycles in the vhost thread, we are saving cycles in the vcpu thread and improving performance. > > 4. vhost statistics > > This patch introduces a set of statistics to monitor different performance > > metrics of vhost and our polling and I/O scheduling mechanisms. The > > statistics are exposed using debugfs and can be easily displayed with a > > Python script (vhost_stat, based on the old kvm_stats) > > https://github.com/abelg/virtual_io_acceleration/commit/ > ac14206ea56939ecc3608dc5f978b86fa322e7b0 > > How about using trace points instead? Besides statistics, it can also > help more in debugging. Yep, we just had a discussion with Gleb about this :) > > > > 5. Add heuristics to improve I/O scheduling > > This patch enhances the round-robin mechanism with a set of heuristics to > > decide when to leave a virtqueue and proceed to the next. > > https://github.com/abelg/virtual_io_acceleration/commit/ > f6a4f1a5d6b82dc754e8af8af327b8d0f043dc4d > > > > This patch improves the handling of the requests by the vhost thread, but > > could perhaps be delayed to a > > later time , and not submitted as one of the first Elvis patches. > > I'd love to hear some comments about whether this patch needs to be part > > of the first submission. > > > > Any other feedback on this plan will be appreciated, > > Thank you, > > Razya > > > > -- > > To unsubscribe from this list: send the line "unsubscribe kvm" in > > the body of a message to majordomo@xxxxxxxxxxxxxxx > > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
