On Thu, 2025-01-16 at 06:50 -0800, Sean Christopherson wrote: > On Thu, Jan 16, 2025, Kai Huang wrote: > > On Mon, 2025-01-13 at 10:09 +0800, Binbin Wu wrote: > > > Lazy check for pending APIC EOI when In-kernel IOAPIC > > > ----------------------------------------------------- > > > In-kernel IOAPIC does not receive EOI with AMD SVM AVIC since the processor > > > accelerates write to APIC EOI register and does not trap if the interrupt > > > is edge-triggered. So there is a workaround by lazy check for pending APIC > > > EOI at the time when setting new IOAPIC irq, and update IOAPIC EOI if no > > > pending APIC EOI. > > > KVM is also not be able to intercept EOI for TDX guests. > > > - When APICv is enabled > > > The code of lazy check for pending APIC EOI doesn't work for TDX because > > > KVM can't get the status of real IRR and ISR, and the values are 0s in > > > vIRR and vISR in apic->regs[], kvm_apic_pending_eoi() will always return > > > false. So the RTC pending EOI will always be cleared when ioapic_set_irq() > > > is called for RTC. Then userspace may miss the coalesced RTC interrupts. > > > - When When APICv is disabled > > > ioapic_lazy_update_eoi() will not be called，then pending EOI status for > > > RTC will not be cleared after setting and this will mislead userspace to > > > see coalesced RTC interrupts. > > > Options: > > > - Force irqchip split for TDX guests to eliminate the use of in-kernel IOAPIC. > > > - Leave it as it is, but the use of RTC may not be accurate. > > > > Looking at the code, it seems KVM only traps EOI for level-triggered interrupt > > for in-kernel IOAPIC chip, but IIUC IOAPIC in userspace also needs to be told > > upon EOI for level-triggered interrupt. I don't know how does KVM works with > > userspace IOAPIC w/o trapping EOI for level-triggered interrupt, but "force > > irqchip split for TDX guest" seems not right. > > Forcing a "split" IRQ chip is correct, in the sense that TDX doesn't support an > I/O APIC and the "split" model is the way to concoct such a setup. With a "full" > IRQ chip, KVM is responsible for emulating the I/O APIC, which is more or less > nonsensical on TDX because it's fully virtual world, i.e. there's no reason to > emulate legacy devices that only know how to talk to the I/O APIC (or PIC, etc.). > Disallowing an in-kernel I/O APIC is ideal from KVM's perspective, because > level-triggered interrupts and thus the I/O APIC as a whole can't be faithfully > emulated (see below). Disabling in-kernel IOAPIC/PIC for TDX guests is fine to me, but I think that, "conceptually", having IOAPIC/PIC in userspace doesn't mean disabling IOAPIC, because theoretically usrespace IOAPIC still needs to be told about the EOI for emulation. I just haven't figured out how does userpsace IOAPIC work with KVM in case of "split IRQCHIP" w/o trapping EOI for level-triggered interrupt. :-) If the point is to disable in-kernel IOAPIC/PIC for TDX guests, then I think both KVM_IRQCHIP_NONE and KVM_IRQCHIP_SPLIT should be allowed for TDX, but not just KVM_IRQCHIP_SPLIT? > > > I think the problem is level-triggered interrupt, > > Yes, because the TDX Module doesn't allow the hypervisor to modify the EOI-bitmap, > i.e. all EOIs are accelerated and never trigger exits. > > > so I think another option is to reject level-triggered interrupt for TDX guest. > > This is a "don't do that, it will hurt" situation. With a sane VMM, the level-ness > of GSIs is controlled by the guest. For GSIs that are routed through the I/O APIC, > the level-ness is determined by the corresponding Redirection Table entry. For > "GSIs" that are actually MSIs (KVM piggybacks legacy GSI routing to let userspace > wire up MSIs), and for direct MSIs injection (KVM_SIGNAL_MSI), the level-ness is > dictated by the MSI itself, which again is guest controlled. > > If the guest induces generation of a level-triggered interrupt, the VMM is left > with the choice of dropping the interrupt, sending it as-is, or converting it to > an edge-triggered interrupt. Ditto for KVM. All of those options will make the > guest unhappy. > > So while it _might_ make debugging broken guests either, I don't think it's worth > the complexity to try and prevent the VMM/guest from sending level-triggered > GSI-routed interrupts. > KVM can at least have some chance to print some error message? > It'd be a bit of a whack-a-mole and there's no architectural > behavior KVM can provide that's better than sending the interrupt and hoping for > the best.
