Christoffer Dall wrote: > > > On Mon, Apr 20, 2009 at 10:56 PM, Oren Laadan <orenl at cs.columbia.edu > <mailto:orenl at cs.columbia.edu>> wrote: > > > > Christoffer Dall wrote: > > see my answers below > > > > (Oren, there's a question hidden in there for you if you have > > time/energy...) > > if you hide better I might even miss it next time :) > > > > > On Mon, Apr 20, 2009 at 8:28 PM, Brian Smith <bls2129 at columbia.edu > <mailto:bls2129 at columbia.edu> > > <mailto:bls2129 at columbia.edu <mailto:bls2129 at columbia.edu>>> wrote: > > > > Hey, > > Sorry, I don't 100% understand, let's take for example the > address > > 0xFFFF0000 (the reset interrupt vector), what will be there > when the > > guest is running? > > > > > > a custom interrupt handler placed above 0xFFFF0000 and mapped by both > > shadow and host page table, which does: > > - replace page table to host page table > > - call host kernel interrupt handler (possibly the KVM handler, which > > you are writing!!!) > > > > > > Will it be the guest's reset interrupt vector, or will it be the > > host's reset interrupt vector. It seems to me it must be the > > latter, but in this case how do we prevent the guest from > > reading/writing to that storage when trying to setup his own > > interrupt handlers? With the guest switch your talking about > > set the page tables to allow access from the appropriate privilege > level only. voila. > > > writing interrupt handlers, I want to make sure you don't > duplicate > > any work I've done, have you looked at my stuff? You said disable > > interrupts, is that until the jump to the guest, or are you > > suggesting the guest should run disabled? If that is the case > then > > there is no point in intercepting IRQ interrupts right? I'm going > > to assume the guest does run enabled so I can build on what > you said > > below to my question. > > > > > > - Of course IRQ's are enabled when running the guest. > > - might not be necessary to disable interrupts, could instead be done > > I'm confused now. disable interrupts when ? of course they must be > kept disabled when a handler is processing an interrupt, for instance. > > and if a new interrupt comes in, whether in host or guest mode, while > the interrupts are disabled, it will be queued. like a regular system. > > > by checking which page table is in effect when handling interrupt - > > probably better. This does not interfere with your work, and we will > > ehhh... I'm not sure this is safe. what are you trying to accomplish ? > > > I was expressing myself poorly. Was thinking that when we switch to the > guest, we want to: > - change the interrupt vectors > - switch the page table > - jump to guest > > if we always code our handlers to replace the page table, then if we got > an interrupt before switching the page table but after changing the > vectors, something weird might happen. On the other hand, if we do it in > the right order, or simply check which page table to make active > depending on the situation we are safe. > > so, the suggestion for disabling interrupts was just to make sure that > the above wouldn't be a problem and was anyway a minor detail:) I > apologize for confusion... yes, definitely: disable interrupts, switch interrupts vector, switch pages table, re-enable interrupts, resume in guest :) > > > > does it at all matter where the interrupt occurred ? (as opposed to > an exception) - interrupts are hardware events that only the host cares > about, so only host handles. on the other hand, guest "interrupts" are > a software creature that is driven by QEMU via well defined interface, > and that you need to arrange to "deliver" to the guest, by looking at > the interrupt vector that the guest (thinks it) has placed. > > > think more about the specifics during the week. We can revisit this > > subject next week if you wish. > > > > > > > > Like I said, obviously we want the host kernel to handle hardware > > interrupts, that isn't what I was asking. I agree 100% that QEMU > > generates the interrupt, and I am guessing it does that > through some > > interface to KVM. If this is the case, we can't do anything with > > the interrupt until the guest looses control and the host (kvm > > module) gains control. That is my question, what do we need to do > > to regain control. > > > > > > I think I understand your question. You have a good point. The > thing we > > have to think about is where to return to from the interrupt. > > > > I suggest (Oren, it would be great if you have input here): > > I think it is helpful to think about "real interrupts" - host hardware > interrupts, and "virtual interrupts" - guest virtual interrupts. the > former is what you need to take care of by the host, and the handlers > should arrange to switch the page tables to the host's tables whenever > they run. (this can be optimized, of course, but probably later). > > the latter, in contrast, are not real interrupts. they are instructions > given by QEMU to KVM, telling KVM to arrange for an interrupt to be > delivered to the guest (from the virtual hardware that the guest uses, > and that is managed by QEMU). > > so what does it mean "arrange for an interrupt" ? nothing to do with > host interrupts. let's think: if the guest were running on the bare > hardware, it would register its interrupt vector somewhere, and an > interrupts would call a callback from that vector. well, because the > guest is virtualized you should be able to intercept when the guest > sets it vector, and _not_ change the real vector, only remember what > the guest requested (while making the guest believe that it succeeded). > > now the next time you are about to resume execution to the guest, you > check if there is an interrupt from QEMU. if there is, then you do > what a hardware interrupt would do: save the hardware context of the > guest (already saved, to some extent, because you were running in the > host mode) - registers, for example, and arrange for the stack to hold > the interrupt data, and then finally call the function registered in > the interrupt vector that you remembered earlier. cleanup when the > guess runs the equivalent of an 'iret'. > > if there isn't an interrupt pending, then simply change to guest mode > without any of these special action. > > now what happened if you are already running in guest mode and QEMU > wants to deliver a "virtual interrupts" to the guest ? well, some > time soon there will be a host timer interrupt, in which your code > will kick in to process whatever, and that code will do the test that > I mentioned above before resuming the guest execution (or it may just > as well context switch to allow another host process to run !). > > > great! I agree 100%. > > > > > - Instead of letting the host return to the guest, let it return > to our > > kvm interrupt handler. > > - In here, if an interrupt has been raised by QEMU since the last > time > > the guest was issued, issue the guest interrupt handler instead of the > > originating code and emulate where to return to in the guest etc. > > - The interrupt would be set asynchronously in QEMU by the signals > > registered on file descripters and a SIG_ALRM for timers. > > - The above can be done by replacing r14 and modifying SPSR before > > calling the host handler. > > > > In this way we let the host kernel deal with nested interrupts and all > > that stuff and the only thing we do is impose a _very_ thin layer > around > > the interrupts and only if we are running a guest. Nice and minimally > > intrusive I think. > > > > > > > > Let me try and ask the question a different way. When any > > asynchronous interrupt occurs the interrupt handler is responsible > > for saving state so when the interrupted task is redispatched it's > > like the interrupt never happened. In the case of an asynchronous > > interrupt while the guest was running, if the only step we > took was > > pass the interrupt to the real handler, at next redispatch the > guest > > will be the one getting control not the host (because the > interrupt > > occurred while running the guest). We don't want that because > then > > we don't have control to issue any interrupts of our own to > the guest. > > Here's the question. Is it sufficient for us to just reenable for > > interrupts in our interrupt handler (host code) and let ourselves > > get interrupted for the same IRQ, but this time the state > saved will > > be the host state and we have successfully broken from the guest. > > Or do we have to give control to the real IRQ handler > ourselves in > > such a way that when it saves state it will give us (host) control > > and not the state of the CPU when the interrupt actually occurred > > (in the guest). "reenable" is easiest to code since nothing needs > > to be done, and this is how ppc works (I believe ppc will keep > > issuing a hardware event until the handler says "I've seen you, > > stop"). I am wondering if anyone knows if that is how ARM > works. I > > think we have to do it the hard way, but I wanted to get your > > opinions on it before attempting it. > > > > UNDEFINED interrupts can legitimately occur if a user program > > issues a coprocessor and the coprocessor doesn't exist (say the > > floating point coprocessor). A good operating system would > emulate > > the user program's intentions if it can. UNDEFINED interrupts can > > also occur if you tried to execute data too. In either case, we > > would want to pass the interrupt to the guest. > > > > I think the terminology for this is "exception" not interrupt. the > difference, for our purpose, is that the latter is synchronous with > the instruction executed by the guess and in response to that specific > instructions. so they need not go through QEMU at all. > > > > > if from host, pass it to host, if from guest, pass it to guest. I > don't > > see any subtleties here. > > > > yup. > > Oren. > > > > > Regards, > > Brian > > > > Christoffer Dall wrote: > > > > Hi there. > > > > Andreas and I are continuing the incremental approach to make > > something running. > > > > The status is this: > > - We can run the bootl oader now, but fast we run into > > problems, since: > > * the kernel expects to be at low address 0x00000008 > > * next step for kernel is to load itself in the upper 1G of > > the virtual memory space > > > > The way forward, which Andreas and I are working on: > > - Create shadow page table > > * have special mapping in upper 64 mb, which Linux > doesn't touch > > * relocate code to this mapping > > - Write guest-switch and interrupt handlers: > > * guest switch will disable interrupts, switch page table, > > switch handlers, jump to guest > > * interrup handler will switch back page table and jump to > > kernel handlers > > > > Regarding the questions below, see my answers inline: > > > > On Mon, Apr 20, 2009 at 12:19 PM, <bls2129 at columbia.edu > <mailto:bls2129 at columbia.edu> > > <mailto:bls2129 at columbia.edu > <mailto:bls2129 at columbia.edu>> <mailto:bls2129 at columbia.edu > <mailto:bls2129 at columbia.edu> > > <mailto:bls2129 at columbia.edu > <mailto:bls2129 at columbia.edu>>>> wrote: > > > > Hi all, > > Here is hopefully a better explanation of what I poorly > > described > > at the meeting today. We have come to agreement that > we need to > > intercept IRQ exceptions, what I am unsure is how to handle > > them. For > > synchronous interrupts we do not pass them to the host > kernel > > handler: > > SWI's that aren't due to translation should be passed along > > to the > > guest, UNDEFINED interrupts that isn't because of a system > > control > > processor operation is handled the same. I am less > certain about > > synchronous abort interrupts, and whether the host kernel > > should see > > these, but that can be a later discussion. > > > > > > I don't understand the above. Can you explain what is the case > > with "UNDEFINED interrupts that isn't because of a system > > control processor operation"? > > > > I am also not certain about how to deal with data aborts. We > > definitely have to wrap logic around it, but the answer will > > come when the guest table is in place the the guest > enables MMU > > and starts managing its own page tables and so on. For > sure data > > aborts should not be injected "raw" into the guest and I don't > > think we need to worry about this right now, but probably next > > week we have to. > > > > > > > > For asynchronous interrupts, we obviously want the host > > kernel to > > handle the interrupt. I was looking at the > architecture and > > what I am > > unsure about is what happens if we are in our IRQ interrupt > > handler > > and enable for interrupts, would the IRQ be reissued by the > > hardware > > or is it lost? If it is reissued that would be great, > > because when > > the host code enables for interrupts, our host state is > saved > > and on > > redispatch we regain control as the host. This is > similar to > > how the > > PPC code works. > > > > > > Hardware interrupts should not be forwarded to guest > kernel. All > > devices are emulated and these interrupts are generated by > QEMU. > > The only thing we have to do is switch the page tables and > > invoke the host handler. (A timer interrupt would send a > SIGALRM > > to qemu, which would eventually cause KVM to emulate a timer > > interrupt and execute the guest handler in this case). > > > > See kvm_arch_pre_run() in target-ppc/kvm.c:121 in the QEMU > > source and kvmppc_check_and_deliver_interrupts() in > > arch/powerpc/kvm/booke_guest.c:209 in the linux kernel. > > > > > > > > If the interrupt would be lost, that means we have to > pass the > > interrupt to the kernel ourselves, but we have to mess > around > > with the > > registers the real IRQ will save to ensure the host regains > > control, > > not the guest (what the registers really were at > interrupt). > > > > Comments/Suggestions? > > > > Brian > > > > _______________________________________________ > > Android-virt mailing list > > Android-virt at lists.cs.columbia.edu > <mailto:Android-virt at lists.cs.columbia.edu> > > <mailto:Android-virt at lists.cs.columbia.edu > <mailto:Android-virt at lists.cs.columbia.edu>> > > <mailto:Android-virt at lists.cs.columbia.edu > <mailto:Android-virt at lists.cs.columbia.edu> > > <mailto:Android-virt at lists.cs.columbia.edu > <mailto:Android-virt at lists.cs.columbia.edu>>> > > > > > https://lists.cs.columbia.edu/cucslists/listinfo/android-virt > > > > > > > ------------------------------------------------------------------------ > > > > > > _______________________________________________ > > Android-virt mailing list > > Android-virt at lists.cs.columbia.edu > <mailto:Android-virt at lists.cs.columbia.edu> > > <mailto:Android-virt at lists.cs.columbia.edu > <mailto:Android-virt at lists.cs.columbia.edu>> > > https://lists.cs.columbia.edu/cucslists/listinfo/android-virt > > > > > > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Android-virt mailing list > > Android-virt at lists.cs.columbia.edu > <mailto:Android-virt at lists.cs.columbia.edu> > > https://lists.cs.columbia.edu/cucslists/listinfo/android-virt > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Android-virt mailing list > Android-virt at lists.cs.columbia.edu > https://lists.cs.columbia.edu/cucslists/listinfo/android-virt
