On Wed, Jun 11, 2014 at 4:49 AM, Zhang Haoyu <zhanghy@xxxxxxxxxxx> wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> According to this: >>>>>>>>> >>>>>>>>> https://github.com/torvalds/linux/commit/0d1de2d901f4ba0972a3886496a44fb1d3300dbd >>>>>>>>> >>>>>>>>> It looks like kvm have been supporting x2apic since kernel 2.6.32, or >>>>>>>>> even earlier. >>>>>>>>> >>>>>>>> This patch is to emulate x2apic for guest, then guest can benefit from some advantages of x2apic, like using RDMSR/WRMSR instead of mmio. >>>>>>>> >>>>>>>>> However, this following patch: >>>>>>>>> >>>>>>>>> https://github.com/torvalds/linux/commit/8d14695f9542e9e0195d6e41ddaa52c32322adf5 >>>>>>>>> >>>>>>>>> Also claims that adding support for x2apic. But this later patch was >>>>>>>>> for kernel 3.9. >>>>>>>> This patch is to support virtual x2apic mode, which can virtualizing MSR-based APIC accesses by configuring MSR bitmaps, >>>>>>>> you can specify some MSR-based accesses without VM exit, the other MSR-based accesses with VM exit, >>>>>>>> which belongs to APIC virtualization from certain angle. >>>>>>> Thanks Haoyu, I am reading the Intel x2apic manual. >>>>>>> >>>>>>> http://www.intel.com/content/dam/doc/specification-update/64-architecture-x2apic-specification.pdf >>>>>>> >>>>>>> but I don't see the so called "virtual x2apic mode", it seems that the >>>>>>> manual does not mention anything about that. Do you mean that when the >>>>>>> Guest OS sets bit 10 (x2apic mode enable bit) of the it virtualized >>>>>>> IA32_APIC_BASE MSR, it is entering a virtual x2apic mode? But isn't >>>>>>> this the same as the aforementioned "emulate x2apic for guest"? Or, >>>>>>> "emulate x2apic for guest" is the foundation of "support virtual >>>>>>> x2apic mode"? >>>>>>> >>>>>>> -Jidong >>>>>>> >>>>>>Oh, I found the answer, the virtualized x2apic mode is not defined in >>>>>>the x2apic manual, but is defined in the intel SDM manual, chapter 29 >>>>>>- APIC virtualization and virtual interrupts. Ignore my questions >>>>>>please, but thank you anyway. >>>>>> >>>>> Yes, virtual x2apic mode is part of APICv. >>>>> Guest has no idea about it is running in a virtual machine, >>>>> VMM also prevent guest from enabling/disabling virtual x2apic mode or some other virtualization configurations. >>>>> >>>> >>>>Regarding this "VMM also prevent guest from enabling/disabling virtual >>>>x2apic mode", do you mean that guest is not allowed to do something >>>>like: >>>> >>>>Assuming we are in the guest, and we see the original value of the >>>>IA32_APICBASE MSR is 0xfee00900, which means EN bit and BSP bit was >>>>enabled. And now, we try to set bit 10 (EXTD bit): >>>> >>>>wrmsr 0x1b 0xfee00d00. >>>> >>>>Do you mean this wrmsr operation is not allowed? If fact, I tried >>>>this, and this simply crashes the guest kernel. (I am using Linux >>>>2.6.34 as the guest kernel, and Linux 3.14 as the host kernel.) And >>>>this is the exact reason that I initialized this whole discussion. My >>>>feeling is this might be a kvm bug, so I wish to figure out and try to >>>>fix it. >>>> >>> The MSR address range 800H through BFFH is architecturally reserved and dedicated for accessing APIC registers in x2APIC mode, >>> you are writing the wrong MSR address. >>> Please see SDN 3A 10.12.1.2 x2APIC Register Address Space, you will get the answer. >>> >>Oh, I was writing to 0x1b, which is the address of IA32_APIC_BASE MSR. >>In particular I was trying to write bit 10, which is the x2APIC >>enable/disable bit. As I said, I just wish to enable that x2APIC. The >>original value 0xfee00900 suggests x2APIC is not enabled, i.e., bit 10 >>is 0. According to SDM manual, this bit has to be 1 if we want to >>enable x2APIC. >> > Sorry, I misread it. > Please try > u64 val; > rdmsrl(0x0000001b, val); > if ( !(val & (1UL << 10)) ) > wrmsrl(0x0000001b, val | (1UL << 10)); > > If the crash still happened, maybe other settings is needed, like switching apic to physical x2apic, I'm not sure. > Please report the crash message. > Yes I wrote a simple kernel module which just includes the above four lines in the module init function, and when I try to install the module with insmod command, the crash still happens, the crash message is: [ 1025.424260] Process bash (pid: 3091, ti=f58b2000 task=f5a725f0 task.ti=f58b2000) [ 1025.425093] Stack: [ 1025.425093] Call Trace: [ 1025.425093] Leftover inexact backtrace: [ 1025.425093] [ 1025.425093] Code: 10 5b 5e 5f 5d e9 98 4a 42 00 8b 0d 3c ba 94 c0 8d 93 f0 00 00 00 89 f0 ff 51 78 eb 07 90 8d 74 26 00 f3 90 ba 80 00 00 00 89 f0 <e8> 12 01 1d 00 85 c0 74 ee eb ad 8d b6 00 00 00 00 8d bf 00 00 And this above message will be displayed repeatedly all the time and the system will become unresponsive. [ 1090.922260] Process bash (pid: 3091, ti=f58b2000 task=f5a725f0 task.ti=f58b2000) [ 1090.923089] Stack: [ 1090.923089] Call Trace: [ 1090.923089] Leftover inexact backtrace: [ 1090.923089] [ 1090.923089] Code: 83 c4 10 5b 5e 5f 5d e9 98 4a 42 00 8b 0d 3c ba 94 c0 8d 93 f0 00 00 00 89 f0 ff 51 78 eb 07 90 8d 74 26 00 f3 90 ba 80 00 00 00 <89> f0 e8 12 01 1d 00 85 c0 74 ee eb ad 8d b6 00 00 00 00 8d bf >>>>>>>>> >>>>>>>>> So I am very confused: >>>>>>>>> >>>>>>>>> First, what's the difference between these two patches? Or say, does >>>>>>>>> kvm support x2apic since kernel 2.6.32, or since kernel 3.9? >>>>>>>>> >>>>>>>> kvm support x2apic since kernel 2.6.32, but not support virtual x2apic mode until kernel 3.9. >>>>>>>> >>>>>>>>> Second, can guest use x2apic even if the host does not? (Assuming qemu >>>>>>>>>has exposed this feature to guest.) The word "use" means something >>>>>>>>> like, accessing x2apic registers, setting the x2apic enable bit in >>>>>>>>> IA32_APICBASE MSR (i.e. bit 10). >>>>>>>>> >>>>>>>> Yes, you can use x2apic even if the PCPU does not support, and benefit from it, like performance bonus from MSR accesses instead of MMIO. >>>>>>>> But, if I remember correctly, if your guest does not support interrupt-remmping, you only can use physical destination mode when using x2apic, >>>>>>>> please see enable_IR_x2apic(). >>>>>>>> >>>>>>>>> -Jidong > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
