On Mon, Apr 27, 2015 at 1:55 PM, Jan Kiszka <jan.kiszka@xxxxxxxxxxx> wrote: > Am 2015-04-27 um 14:35 schrieb Jan Kiszka: >> Am 2015-04-27 um 12:17 schrieb Stefan Hajnoczi: >>> On Sun, Apr 26, 2015 at 2:24 PM, Luke Gorrie <luke@xxxxxxxx> wrote: >>>> On 24 April 2015 at 15:22, Stefan Hajnoczi <stefanha@xxxxxxxxx> wrote: >>>>> >>>>> The motivation for making VM-to-VM fast is that while software >>>>> switches on the host are efficient today (thanks to vhost-user), there >>>>> is no efficient solution if the software switch is a VM. >>>> >>>> >>>> I see. This sounds like a noble goal indeed. I would love to run the >>>> software switch as just another VM in the long term. It would make it much >>>> easier for the various software switches to coexist in the world. >>>> >>>> The main technical risk I see in this proposal is that eliminating the >>>> memory copies might not have the desired effect. I might be tempted to keep >>>> the copies but prevent the kernel from having to inspect the vrings (more >>>> like vhost-user). But that is just a hunch and I suppose the first step >>>> would be a prototype to check the performance anyway. >>>> >>>> For what it is worth here is my view of networking performance on x86 in the >>>> Haswell+ era: >>>> https://groups.google.com/forum/#!topic/snabb-devel/aez4pEnd4ow >>> >>> Thanks. >>> >>> I've been thinking about how to eliminate the VM <-> host <-> VM >>> switching and instead achieve just VM <-> VM. >>> >>> The holy grail of VM-to-VM networking is an exitless I/O path. In >>> other words, packets can be transferred between VMs without any >>> vmexits (this requires a polling driver). >>> >>> Here is how it works. QEMU gets "-device vhost-user" so that a VM can >>> act as the vhost-user server: >>> >>> VM1 (virtio-net guest driver) <-> VM2 (vhost-user device) >>> >>> VM1 has a regular virtio-net PCI device. VM2 has a vhost-user device >>> and plays the host role instead of the normal virtio-net guest driver >>> role. >>> >>> The ugly thing about this is that VM2 needs to map all of VM1's guest >>> RAM so it can access the vrings and packet data. The solution to this >>> is something like the Shared Buffers BAR but this time it contains not >>> just the packet data but also the vring, let's call it the Shared >>> Virtqueues BAR. >>> >>> The Shared Virtqueues BAR eliminates the need for vhost-net on the >>> host because VM1 and VM2 communicate directly using virtqueue notify >>> or polling vring memory. Virtqueue notify works by connecting an >>> eventfd as ioeventfd in VM1 and irqfd in VM2. And VM2 would also have >>> an ioeventfd that is irqfd for VM1 to signal completions. >> >> We had such a discussion before: >> http://thread.gmane.org/gmane.comp.emulators.kvm.devel/123014/focus=279658 >> >> Would be great to get this ball rolling again. >> >> Jan >> > > But one challenge would remain even then (unless both sides only poll): > exit-free inter-VM signaling, no? But that's a hardware issue first of all. To start with ioeventfd<->irqfd can be used. It incurs a light-weight exit in VM1 and interrupt injection in VM2. For networking the cost is mitigated by NAPI drivers which switch between interrupts and polling. During notification-heavy periods the guests would use polling anyway. A hardware solution would be some kind of inter-guest interrupt injection. I don't know VMX well enough to know whether that is possible on Intel CPUs. Stefan _______________________________________________ Virtualization mailing list Virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx https://lists.linuxfoundation.org/mailman/listinfo/virtualization
