On Fri, 2012-10-26 at 12:17 +0100, Peter Maydell wrote: > > Well, that's the thing, I haven't managed to figure that out so far, > it > > looks very x86-specific to me. To begin with there's no such thing > as a > > "GSI" in our world. > > This was roughly the feeling I had looking at these APIs. There > might be some underlying generic concept but there is a definite > tendency for the surface representation to use x86 specific > terminology to the extent that you can't tell whether an API > is x86 specific or merely apparently so... Right. Which is why I'm sure I'm actually missing something there :-) And I'm hoping Paolo and Jan will help shed some light. It might help if somebody could explain a bit more what a GSI is in x86 land and how it relates to the various APICs, along with what exactly they mean by "routing" , ie. what are the different elements that get associated. Basically, if somebody could describe how the x86 stuff works, that might help. >From my view of things, we have various "sources" of interrupts. On my list are emulated device LSIs, emulated device MSIs, both in qemu, then vhost and finally pass-through, I suppose on some platforms IPIs come in as well though. Those "sources" need, one way or another, to hit a source controller which will then itself, in a platform specific way, shoot the interrupt to a presentation controller. The routing between source and presentation controllers is fairly platform specific as far as I can tell even within a given CPU family. Ie the way an OpenPIC (aka MPIC, used on macs) does it is different than the way the XICS system does it on pseries, and is different from most embedded stuff (which typically doesn't have that source/presentation distinction but just cascaded dumber PICs). The amount of configurability, the type of configuration information etc... that governs such a layout is also very specific to the platform and the type of interrupt controller system used on it. Remains the "routing" between source of "events" and actual "inputs" to a source controller. This too doesn't seem totally obvious to generalize. For example an embedded platform with a bunch of cascaded dumb interrupt controllers doesn't have a concept of a flat number space in HW, an interrupt "input" to be identified properly, needs to identify the controller and the interrupt within that controller. However, within KVM/qemu, it's pretty easy to assign to each controller a number and by collating the two, get some kind of flat space, though it's not arbitrary and the routing is thus fairly constrained if not totally fixed. In the pseries case, the global number is split in two bit fields, the BUID identifying the specific source controller and the source within that controller. Here too it's fairly fixed though. So the ioctl we use to create a source controller in the kernel takes the BUID as an argument, and from there the kernel will "find" the right source controller based solely on the interrupt number. So basically on one side we have a global interrupt number that identifies an "input", I assume that's what x86 calls a GSI ? Remains how to associate the various sources of interrupts to that 'global number'... and that is fairly specific to each source type isn't it ? In our current powerpc code, the emulated devices toggle the qirq which ends up shooting an ioctl to set/reset or "message" (for MSIs) the corresponding global interrupt. The mapping is established entirely within qemu, we just tell the kernel to trigger a given interrupt. We haven't really sorted vhost out yet so I'm not sure how that will work out but the idea would be to have an ioctl to associate an eventfd or whatever vhost uses as interrupt "outputs" with a global interrupt number. For pass-through, currently our VFIO is dumb, interrupts get to qemu which then shoots them back to the kernel using the standard qirq stuff used by emulated devices. Here I suppose we would want something similar to vhost to associate the VFIO irq fd with a "global number". Is that what the existing ioctl's provide ? Their semantics aren't totally obvious to me. Note that for pass-through at least, and possibly for vhost, we'd like to actually totally bypass the irqfd & eventfd stuff for performance reasons. At least for VFIO, if we are going to get the max performance out of it, we need to take all generic code out of the picture. IE. If the interrupts are routed to the physical CPU where the guest is running, we want to be able to catch and distribute the interrupts to the guest entirely within guest context, ie, with KVM arch specific low level code that runs in "real mode" (ie MMU off) without context switching the MMU back to the host, which on POWER is fairly costly. That means that at least the association between a guest global interrupt number and a host global interrupt number for pass-through will be something that goes entirely through arch specific code path. We might still be able to use generic APIs to establish it if they are suitable though. Cheers, Ben. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
