Radim, On Thu, Feb 12, 2015 at 07:41:30PM +0100, Radim Krčmář wrote: > Each patch has a diff from v1, here is only a prologue on the mythical > mixed xAPIC and x2APIC mode: > > There is one interesting alias in xAPIC and x2APIC ICR destination, the > 0xff000000, which is a broadcast in xAPIC and either a physical message > to high x2APIC ID or a message to an empty set in x2APIC cluster. > > This corner case in mixed mode can be triggered by a weird message > 1) when we have no x2APIC ID 0xff000000, but send x2APIC message there 10.7 SYSTEM AND APIC BUS ARBITRATION Note that except for the SIPI IPI (see Section 10.6.1, “Interrupt Command Register (ICR)”), all bus messages that fail to be delivered to their specified destination or destinations are automatically retried. Software should avoid situations in which IPIs are sent to disabled or nonexistent local APICs, causing the messages to be resent repeatedly. > or after something that isn't forbidden in SDM most likely because they > didn't think that anyone would ever consider it > 2) we have x2APIC ID 0xff000000 and reset other x2APICs into xAPIC mode Or just x2APIC initialization in non lockstep: vcpu0 vcpu1 T0) xapic mode xapic mode T1) x2apic enable T2) broadcast IPI > Current KVM doesn't need to consider (2), so there only is a slim chance > that some hobbyist OS pushes the specification to its limits. > > The problem is that SDM for xAPIC doesn't believably specify[1] if all > messages beginning with 0xff are considered as broadcasts, 10.6.2.1: > A broadcast IPI (bits 28-31 of the MDA are 1's) > > No volunteer came to clear this up, so I hacked Linux to have one x2APIC > core between xAPIC ones. Physical IPI to 0xff000000 got delivered only > to CPU0, like most other destinations, Logical IPI to 0xff000000 got > dropped and only 0xffffffff worked as a broadcast in both modes. > > I think it is because Intel never considered mixed mode to be valid, and > seen delivery might be an emergent feature of QPI routing. > Luckily, broadcast from xAPIC isn't delivered to x2APIC. In real hardware? > Real exploration would require greater modifications to Linux (ideally > writing a custom kernel), so this series implements something that makes > some sense and isn't too far from reality. Ok, so the problem is that broadcast (ICR.destination == 0xff000000) from xAPIC CPU is not delivered to x2APIC CPUs ? > Radim Krčmář (4): > KVM: x86: use MDA for interrupt matching > KVM: x86: fix mixed APIC mode broadcast > KVM: x86: avoid logical_map when it is invalid > KVM: x86: simplify kvm_apic_map I can't find any restriction against (cpu0==x2APIC, cpu1==xAPIC) in the documentation. Anyway, emulation should match physical hardware. From your message above, it is not clear what is the behaviour there: "No volunteer came to clear this up, so I hacked Linux to have one x2APIC core between xAPIC ones. Physical IPI to 0xff000000 got delivered only to CPU0, like most other destinations, Logical IPI to 0xff000000 got dropped and only 0xffffffff worked as a broadcast in both modes. Luckily, broadcast from xAPIC isn't delivered to x2APIC." >From the x2APIC CPU or the xAPIC ones? It should be easy to write kvm-unit-test testcases that match physical hardware behaviour (in general, i am having a hard time figure out in what way "mixed mode" is supposed to behave, please describe it clearly). -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
