On Wed, 2010-10-27 at 11:23 -0700, Etienne Martineau wrote: > On Wed, 27 Oct 2010, Alex Williamson wrote: > > I've actually been working on porting the qemu vfio driver over to > > qemu-kvm recently, and nearly have it working. For MSI-X interrupts > > I've ported to the common msix.c code, which abstracts how the interrupt > > is actually sent to the guest. I've also added irqfd support in the > > msix mask notifier so MSI-X interrupts avoid bouncing through qemu. MSI > > support should work the same once Michael's msi.c is upstream. > > > > For INTx interrupts, qemu_set_irq will also work with KVM (it has to or > > all of the emulated drivers would break). The problem is getting an EOI > > back from the KVM kernel apic. I'm currently working on code that adds > > a new KVM ioctl to register an eventfd for the EOI, which then triggers > > qemu-kvm to re-enable the interrupt. My hope is that we can add irqfd > > support to both of these paths, so INTx is injected directly from VFIO > > into KVM, and VFIO can directly consume the KVM EOI. > > OK let me try to understand what you've done (please correct me if I'm > wrong). Emulated devices relies on 'kvm_irqfd' for interrupts delivery. No, emulated devices trigger interrupts directly with qemu_set_irq. irqfds are currently only used by vhost afaik, since it's being interrupted externally, much like pass through devices are. > Somehow you've modify VFIO to understand 'kvm_irqfd' so that whenever the > assigned devices receive an IRQ it pass it directly to kvm without > bouncing to userspace? Sort of. When the VFIO device triggers an interrupt, we get notified via the eventfd we've registered for that interrupt. We can then call qemu_set_irq directly to raise that interrupt in the KVM kernel APIC. That much works today. The irqfd mechanism is simply a way for KVM to directly consume the eventfd and raise an interrupt via a pre-setup vector. That's yet to be implemented for INTx on VFIO, but should mostly be a matter of connecting existing pieces together. It's working for MSI-X. > I'm not sure to understand the part where VFIO signal back the EIO to KVM? When VFIO sends an interrupt, it disables the physical device from generating more interrupts (this is where VFIO requires PCI 2.3 compliant devices for the INTx disable bit int he status register). When the guest services the interrupt, we can detect this by catching the EOI of the IOAPIC. At that point, we can re-eanble interrupts on the device. Wash, rinse, repeat. To do this in qemu, I created a callback on the ioapic where drivers can register for the interrupt they care about. Since KVM moves the ioapic into the kernel, we need to extend this into KVM and have yet another eventfd mechanism. It's possible that we could have the VFIO kernel module also receive this eventfd, re-enabling interrupts on the device, in much the same way as above. I haven't tried this yet, but it should just be a matter of creating another VFIO ioctl and stealing code from the KVM irqfd setup. > Also, with your change do you think that VFIO can be keept generic? > Reason I'm asking is because we are plannig to use VFIO for some userspace > drivers... Yes, none of this requires KVM specific modifications to VFIO. VFIO is still just triggering eventfds, and hopefully receiving one via an irqfd-like mechanism for EOI. > >> Because qemu device assignment is working on VFIO I'm making the > >> assumption that kvm iommu code can be entirely deprecated. Maybe I'm > >> totally wrong here? > > > > Yes, VFIO makes no use of it. > > Yes I'm wrong? VFIO does not make any use of the KVM iommu code, or any of the KVM device assignment ioctls for that matter. Thanks, Alex -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
