On Mon, May 5, 2014 at 4:52 PM, Eric Auger <eric.auger@xxxxxxxxxx> wrote: > > Dear All, > > I will try to sketch a global view of how to assign a physical IRQ to a > KVM guest using the following subsystems: > > - on kernel side: VFIO driver, KVM IRQFD and GSI routing On the kernel side there are two sides: 1) The VFIO eventfd support which is underway for v6, as mentioned in the VFIO patches. 2) KVM IRQFD support, which depends on GSI routing as we discussed here. For (1) it is not a different story than what VFIO already does on x86. We are going to support the same API already documented in include/uapi/linux/vfio.h and will be part of the next version of the VFIO_PLATFORM patches. For (2) we would like to offer an RFC patch, as soon as we have some kind of agreement with that what we have started is the right direction and desirable, which is why we started this thread. > - on user side: QEMU system image featuring VFIO QEMU device. > > It aims at sharing knowledge and checking that the understanding of the > legacy is correct (MSI routing is out of scope). > > GSI routing table: > > Each VM has its own routing table. This later aims at storing how a > physical IRQ (the gsi) is connected to a guest. > > GSI routing entries contain the following fields (not exhaustive): > - gsi (the physical IRQ) > - irqchip (the virtual interrupt controller) > - irqchip.pin (the interrupt controller input the gsi is routed to). > - set() is the method that enables to trigger the virtual interrupt for > this entry (basically depends on the irqchip, IOAPI, PIC, GIC, ...). > The complete definition can be found in include/linux/kvm_host.h. set() is internal to the kernel and will not be exposed to userspace (see discussions on the API previously in the thread) > > IRQFD: > > irqfd framework makes it possible to assign physical IRQs (the gsi > above) to KVM guests. An eventfd is associated to a physical IRQ (the > gsi). When the eventfd is signaled (typically by a VFIO driver ISR), the > irqfd framework has the role to inject the virtual IRQ associated to > this physical IRQ. This is done on kernel side. The injection is made > through the virtual interrupt controller and made visible on next VM > runtime window. > > the _irqfd struct (eventfd.c) itself stores the VM it applies to and the > gsi it is linked with. > > The irqfd framework depends on the KVM routing table to find the > remaining information needed to perform the guest injection: > - the virtual interrupt controller used for injection (irqchip) > - the input pin of the interrupt controller (irqchip.pin) > - the set() function irdfq must call to trigger the virtual IRQ > > Although the _irqfd struct has a routing entry field (irq_entry), this > one is not used for GSI (it is for MSI). Thus when the eventfd is > signaled, the routing table is searched for the associated GSI and > eventually set() is called. > > Note that irqfd allows you to define a second (optional) eventfd (called > the resampler), which can be signaled by KVM when the virtual IRQ is > completed (EOI). The resamplerfd part is actually very important to properly support level sensitive (automasked) interrupts. For edge triggered interrupts IRQFD is just a matter of efficiency, but level-sensitive interrupts absolutely need a IRQFD and a RESAMPLEFD to be implemented in a sane way. Luckily, when we properly implement the irqchip.c interface and IRQ routing with the VGIC, it will give us all that almost for free! The eventfd code is already decoupled from architecture specific code. > > QEMU side: > > The irq routing table can be set using KVM user API using > KVM_SET_GSI_ROUTING. Typically this would be QEMU's job at init, after > which QEMU would not be involved anymore in IRQ injection. > > By the way it is also interesting to note that KVM_ASSIGN_DEV_IRQ makes > it possible to define a correspondance between a host physical IRQ and a > guest virtual IRQ. But it relies on PCI device assignment. This API does > not support MSI which is a drawback compared to IRQFD one. It sounds to > me that implementing irqfd and GSI routing looks more straightforward > and maybe more future proof. And what if KVM_ASSIGN_DEV_IRQ becomes deprecated in the future? For this reason we are implementing the "VFIO way", which is proper IRQFD support (with GSI routing as a straight dependency). > > > ARM PORTING: > > Let's consider how to apply this in the ARM world. On ARM we only have > a single kind of irqchip, namely the VGIC. the virtual interrupt > controller is implemented in virt/kvm/arm/vgic.c. > > irq_comm.c still contains quite a lot of architecture specific code and > needs to be retargeted for ARM as it is today for x86 or IA64. For > example kvm_set_pic_irq need to call kvm_vgic_inject_vfio_irq. > CONFIG_ARM_GIC could be used to #ifdef the code. An alternative would be > to refactor the code into architecture specific files. This has already been mentioned at the beginning of the thread. irq_comm.c contains architecture specific code, but irqchip.c does not! Architecture dependent parts are already neatly separated. For example arch/powerpc/kvm/mpic.c already implements IRQ routing without touching irq_comm.c. We plan to do the same with the VGIC. > > My current understanding is that kvm irqchip and irq routing only is > used by irqfd and assigned_dev. I am not sure whether there is any > interest in supporting injection of other IRQs than SPI (SGI, PPI are > not supposed to be routed by either irqfd, assigned dev, is that a > correct understanding?). > SGIs are not supposed to be injected, but I don't know if we can push PPIs under the rug... > Please feel free to correct and comment. > > Thank you in advance > > Best Regards > > Eric > _______________________________________________ kvmarm mailing list kvmarm@xxxxxxxxxxxxxxxxxxxxx https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
