Christoffer Dall wrote: > see my answers below > > (Oren, there's a question hidden in there for you if you have > time/energy...) if you hide better I might even miss it next time :) > > On Mon, Apr 20, 2009 at 8:28 PM, Brian Smith <bls2129 at columbia.edu > <mailto:bls2129 at columbia.edu>> wrote: > > Hey, > Sorry, I don't 100% understand, let's take for example the address > 0xFFFF0000 (the reset interrupt vector), what will be there when the > guest is running? > > > a custom interrupt handler placed above 0xFFFF0000 and mapped by both > shadow and host page table, which does: > - replace page table to host page table > - call host kernel interrupt handler (possibly the KVM handler, which > you are writing!!!) > > > Will it be the guest's reset interrupt vector, or will it be the > host's reset interrupt vector. It seems to me it must be the > latter, but in this case how do we prevent the guest from > reading/writing to that storage when trying to setup his own > interrupt handlers? With the guest switch your talking about set the page tables to allow access from the appropriate privilege level only. voila. > writing interrupt handlers, I want to make sure you don't duplicate > any work I've done, have you looked at my stuff? You said disable > interrupts, is that until the jump to the guest, or are you > suggesting the guest should run disabled? If that is the case then > there is no point in intercepting IRQ interrupts right? I'm going > to assume the guest does run enabled so I can build on what you said > below to my question. > > > - Of course IRQ's are enabled when running the guest. > - might not be necessary to disable interrupts, could instead be done I'm confused now. disable interrupts when ? of course they must be kept disabled when a handler is processing an interrupt, for instance. and if a new interrupt comes in, whether in host or guest mode, while the interrupts are disabled, it will be queued. like a regular system. > by checking which page table is in effect when handling interrupt - > probably better. This does not interfere with your work, and we will ehhh... I'm not sure this is safe. what are you trying to accomplish ? does it at all matter where the interrupt occurred ? (as opposed to an exception) - interrupts are hardware events that only the host cares about, so only host handles. on the other hand, guest "interrupts" are a software creature that is driven by QEMU via well defined interface, and that you need to arrange to "deliver" to the guest, by looking at the interrupt vector that the guest (thinks it) has placed. > think more about the specifics during the week. We can revisit this > subject next week if you wish. > > > > Like I said, obviously we want the host kernel to handle hardware > interrupts, that isn't what I was asking. I agree 100% that QEMU > generates the interrupt, and I am guessing it does that through some > interface to KVM. If this is the case, we can't do anything with > the interrupt until the guest looses control and the host (kvm > module) gains control. That is my question, what do we need to do > to regain control. > > > I think I understand your question. You have a good point. The thing we > have to think about is where to return to from the interrupt. > > I suggest (Oren, it would be great if you have input here): I think it is helpful to think about "real interrupts" - host hardware interrupts, and "virtual interrupts" - guest virtual interrupts. the former is what you need to take care of by the host, and the handlers should arrange to switch the page tables to the host's tables whenever they run. (this can be optimized, of course, but probably later). the latter, in contrast, are not real interrupts. they are instructions given by QEMU to KVM, telling KVM to arrange for an interrupt to be delivered to the guest (from the virtual hardware that the guest uses, and that is managed by QEMU). so what does it mean "arrange for an interrupt" ? nothing to do with host interrupts. let's think: if the guest were running on the bare hardware, it would register its interrupt vector somewhere, and an interrupts would call a callback from that vector. well, because the guest is virtualized you should be able to intercept when the guest sets it vector, and _not_ change the real vector, only remember what the guest requested (while making the guest believe that it succeeded). now the next time you are about to resume execution to the guest, you check if there is an interrupt from QEMU. if there is, then you do what a hardware interrupt would do: save the hardware context of the guest (already saved, to some extent, because you were running in the host mode) - registers, for example, and arrange for the stack to hold the interrupt data, and then finally call the function registered in the interrupt vector that you remembered earlier. cleanup when the guess runs the equivalent of an 'iret'. if there isn't an interrupt pending, then simply change to guest mode without any of these special action. now what happened if you are already running in guest mode and QEMU wants to deliver a "virtual interrupts" to the guest ? well, some time soon there will be a host timer interrupt, in which your code will kick in to process whatever, and that code will do the test that I mentioned above before resuming the guest execution (or it may just as well context switch to allow another host process to run !). > - Instead of letting the host return to the guest, let it return to our > kvm interrupt handler. > - In here, if an interrupt has been raised by QEMU since the last time > the guest was issued, issue the guest interrupt handler instead of the > originating code and emulate where to return to in the guest etc. > - The interrupt would be set asynchronously in QEMU by the signals > registered on file descripters and a SIG_ALRM for timers. > - The above can be done by replacing r14 and modifying SPSR before > calling the host handler. > > In this way we let the host kernel deal with nested interrupts and all > that stuff and the only thing we do is impose a _very_ thin layer around > the interrupts and only if we are running a guest. Nice and minimally > intrusive I think. > > > > Let me try and ask the question a different way. When any > asynchronous interrupt occurs the interrupt handler is responsible > for saving state so when the interrupted task is redispatched it's > like the interrupt never happened. In the case of an asynchronous > interrupt while the guest was running, if the only step we took was > pass the interrupt to the real handler, at next redispatch the guest > will be the one getting control not the host (because the interrupt > occurred while running the guest). We don't want that because then > we don't have control to issue any interrupts of our own to the guest. > Here's the question. Is it sufficient for us to just reenable for > interrupts in our interrupt handler (host code) and let ourselves > get interrupted for the same IRQ, but this time the state saved will > be the host state and we have successfully broken from the guest. > Or do we have to give control to the real IRQ handler ourselves in > such a way that when it saves state it will give us (host) control > and not the state of the CPU when the interrupt actually occurred > (in the guest). "reenable" is easiest to code since nothing needs > to be done, and this is how ppc works (I believe ppc will keep > issuing a hardware event until the handler says "I've seen you, > stop"). I am wondering if anyone knows if that is how ARM works. I > think we have to do it the hard way, but I wanted to get your > opinions on it before attempting it. > > UNDEFINED interrupts can legitimately occur if a user program > issues a coprocessor and the coprocessor doesn't exist (say the > floating point coprocessor). A good operating system would emulate > the user program's intentions if it can. UNDEFINED interrupts can > also occur if you tried to execute data too. In either case, we > would want to pass the interrupt to the guest. > I think the terminology for this is "exception" not interrupt. the difference, for our purpose, is that the latter is synchronous with the instruction executed by the guess and in response to that specific instructions. so they need not go through QEMU at all. > > if from host, pass it to host, if from guest, pass it to guest. I don't > see any subtleties here. > yup. Oren. > > Regards, > Brian > > Christoffer Dall wrote: > > Hi there. > > Andreas and I are continuing the incremental approach to make > something running. > > The status is this: > - We can run the bootl oader now, but fast we run into > problems, since: > * the kernel expects to be at low address 0x00000008 > * next step for kernel is to load itself in the upper 1G of > the virtual memory space > > The way forward, which Andreas and I are working on: > - Create shadow page table > * have special mapping in upper 64 mb, which Linux doesn't touch > * relocate code to this mapping > - Write guest-switch and interrupt handlers: > * guest switch will disable interrupts, switch page table, > switch handlers, jump to guest > * interrup handler will switch back page table and jump to > kernel handlers > > Regarding the questions below, see my answers inline: > > On Mon, Apr 20, 2009 at 12:19 PM, <bls2129 at columbia.edu > <mailto:bls2129 at columbia.edu> <mailto:bls2129 at columbia.edu > <mailto:bls2129 at columbia.edu>>> wrote: > > Hi all, > Here is hopefully a better explanation of what I poorly > described > at the meeting today. We have come to agreement that we need to > intercept IRQ exceptions, what I am unsure is how to handle > them. For > synchronous interrupts we do not pass them to the host kernel > handler: > SWI's that aren't due to translation should be passed along > to the > guest, UNDEFINED interrupts that isn't because of a system > control > processor operation is handled the same. I am less certain about > synchronous abort interrupts, and whether the host kernel > should see > these, but that can be a later discussion. > > > I don't understand the above. Can you explain what is the case > with "UNDEFINED interrupts that isn't because of a system > control processor operation"? > > I am also not certain about how to deal with data aborts. We > definitely have to wrap logic around it, but the answer will > come when the guest table is in place the the guest enables MMU > and starts managing its own page tables and so on. For sure data > aborts should not be injected "raw" into the guest and I don't > think we need to worry about this right now, but probably next > week we have to. > > > > For asynchronous interrupts, we obviously want the host > kernel to > handle the interrupt. I was looking at the architecture and > what I am > unsure about is what happens if we are in our IRQ interrupt > handler > and enable for interrupts, would the IRQ be reissued by the > hardware > or is it lost? If it is reissued that would be great, > because when > the host code enables for interrupts, our host state is saved > and on > redispatch we regain control as the host. This is similar to > how the > PPC code works. > > > Hardware interrupts should not be forwarded to guest kernel. All > devices are emulated and these interrupts are generated by QEMU. > The only thing we have to do is switch the page tables and > invoke the host handler. (A timer interrupt would send a SIGALRM > to qemu, which would eventually cause KVM to emulate a timer > interrupt and execute the guest handler in this case). > > See kvm_arch_pre_run() in target-ppc/kvm.c:121 in the QEMU > source and kvmppc_check_and_deliver_interrupts() in > arch/powerpc/kvm/booke_guest.c:209 in the linux kernel. > > > > If the interrupt would be lost, that means we have to pass the > interrupt to the kernel ourselves, but we have to mess around > with the > registers the real IRQ will save to ensure the host regains > control, > not the guest (what the registers really were at interrupt). > > Comments/Suggestions? > > Brian > > _______________________________________________ > Android-virt mailing list > Android-virt at lists.cs.columbia.edu > <mailto:Android-virt at lists.cs.columbia.edu> > <mailto:Android-virt at lists.cs.columbia.edu > <mailto:Android-virt at lists.cs.columbia.edu>> > > https://lists.cs.columbia.edu/cucslists/listinfo/android-virt > > > ------------------------------------------------------------------------ > > > _______________________________________________ > Android-virt mailing list > Android-virt at lists.cs.columbia.edu > <mailto:Android-virt at lists.cs.columbia.edu> > https://lists.cs.columbia.edu/cucslists/listinfo/android-virt > > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Android-virt mailing list > Android-virt at lists.cs.columbia.edu > https://lists.cs.columbia.edu/cucslists/listinfo/android-virt
