>>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> According to this: >>>>>>>>>> >>>>>>>>>> https://github.com/torvalds/linux/commit/0d1de2d901f4ba0972a3886496a44fb1d3300dbd >>>>>>>>>> >>>>>>>>>> It looks like kvm have been supporting x2apic since kernel 2.6.32, or >>>>>>>>>> even earlier. >>>>>>>>>> >>>>>>>>> This patch is to emulate x2apic for guest, then guest can benefit from some advantages of x2apic, like using RDMSR/WRMSR instead of mmio. >>>>>>>>> >>>>>>>>>> However, this following patch: >>>>>>>>>> >>>>>>>>>> https://github.com/torvalds/linux/commit/8d14695f9542e9e0195d6e41ddaa52c32322adf5 >>>>>>>>>> >>>>>>>>>> Also claims that adding support for x2apic. But this later patch was >>>>>>>>>> for kernel 3.9. >>>>>>>>> This patch is to support virtual x2apic mode, which can virtualizing MSR-based APIC accesses by configuring MSR bitmaps, >>>>>>>>> you can specify some MSR-based accesses without VM exit, the other MSR-based accesses with VM exit, >>>>>>>>> which belongs to APIC virtualization from certain angle. >>>>>>>> Thanks Haoyu, I am reading the Intel x2apic manual. >>>>>>>> >>>>>>>> http://www.intel.com/content/dam/doc/specification-update/64-architecture-x2apic-specification.pdf >>>>>>>> >>>>>>>> but I don't see the so called "virtual x2apic mode", it seems that the >>>>>>>> manual does not mention anything about that. Do you mean that when the >>>>>>>> Guest OS sets bit 10 (x2apic mode enable bit) of the it virtualized >>>>>>>> IA32_APIC_BASE MSR, it is entering a virtual x2apic mode? But isn't >>>>>>>> this the same as the aforementioned "emulate x2apic for guest"? Or, >>>>>>>> "emulate x2apic for guest" is the foundation of "support virtual >>>>>>>> x2apic mode"? >>>>>>>> >>>>>>>> -Jidong >>>>>>>> >>>>>>>Oh, I found the answer, the virtualized x2apic mode is not defined in >>>>>>>the x2apic manual, but is defined in the intel SDM manual, chapter 29 >>>>>>>- APIC virtualization and virtual interrupts. Ignore my questions >>>>>>>please, but thank you anyway. >>>>>>> >>>>>> Yes, virtual x2apic mode is part of APICv. >>>>>> Guest has no idea about it is running in a virtual machine, >>>>>> VMM also prevent guest from enabling/disabling virtual x2apic mode or some other virtualization configurations. >>>>>> >>>>> >>>>>Regarding this "VMM also prevent guest from enabling/disabling virtual >>>>>x2apic mode", do you mean that guest is not allowed to do something >>>>>like: >>>>> >>>>>Assuming we are in the guest, and we see the original value of the >>>>>IA32_APICBASE MSR is 0xfee00900, which means EN bit and BSP bit was >>>>>enabled. And now, we try to set bit 10 (EXTD bit): >>>>> >>>>>wrmsr 0x1b 0xfee00d00. >>>>> >>>>>Do you mean this wrmsr operation is not allowed? If fact, I tried >>>>>this, and this simply crashes the guest kernel. (I am using Linux >>>>>2.6.34 as the guest kernel, and Linux 3.14 as the host kernel.) And >>>>>this is the exact reason that I initialized this whole discussion. My >>>>>feeling is this might be a kvm bug, so I wish to figure out and try to >>>>>fix it. >>>>> >>>> The MSR address range 800H through BFFH is architecturally reserved and dedicated for accessing APIC registers in x2APIC mode, >>>> you are writing the wrong MSR address. >>>> Please see SDN 3A 10.12.1.2 x2APIC Register Address Space, you will get the answer. >>>> >>>Oh, I was writing to 0x1b, which is the address of IA32_APIC_BASE MSR. >>>In particular I was trying to write bit 10, which is the x2APIC >>>enable/disable bit. As I said, I just wish to enable that x2APIC. The >>>original value 0xfee00900 suggests x2APIC is not enabled, i.e., bit 10 >>>is 0. According to SDM manual, this bit has to be 1 if we want to >>>enable x2APIC. >>> >> Sorry, I misread it. >> Please try >> u64 val; >> rdmsrl(0x0000001b, val); >> if ( !(val & (1UL << 10)) ) >> wrmsrl(0x0000001b, val | (1UL << 10)); >> >> If the crash still happened, maybe other settings is needed, like switching apic to physical x2apic, I'm not sure. >> Please report the crash message. >> >Yes I wrote a simple kernel module which just includes the above four >lines in the module init function, and when I try to install the >module with insmod command, the crash still happens, the crash message >is: > >[ 1025.424260] Process bash (pid: 3091, ti=f58b2000 task=f5a725f0 >task.ti=f58b2000) >[ 1025.425093] Stack: >[ 1025.425093] Call Trace: >[ 1025.425093] Leftover inexact backtrace: >[ 1025.425093] >[ 1025.425093] Code: 10 5b 5e 5f 5d e9 98 4a 42 00 8b 0d 3c ba 94 c0 >8d 93 f0 00 00 00 89 f0 ff 51 78 eb 07 90 8d 74 26 00 f3 90 ba 80 00 >00 00 89 f0 <e8> 12 01 1d 00 85 c0 74 ee eb ad 8d b6 00 00 00 00 8d bf >00 00 > Does the vcpu have x2apic feature? please make a confirmation by 'cat /proc/cpuinfo | grep -i x2apic' Configure crashkernel to kdump more info, please. Thanks, Zhang Haoyu >And this above message will be displayed repeatedly all the time and >the system will become unresponsive. >[ 1090.922260] Process bash (pid: 3091, ti=f58b2000 task=f5a725f0 >task.ti=f58b2000) >[ 1090.923089] Stack: >[ 1090.923089] Call Trace: >[ 1090.923089] Leftover inexact backtrace: >[ 1090.923089] >[ 1090.923089] Code: 83 c4 10 5b 5e 5f 5d e9 98 4a 42 00 8b 0d 3c ba >94 c0 8d 93 f0 00 00 00 89 f0 ff 51 78 eb 07 90 8d 74 26 00 f3 90 ba >80 00 00 00 <89> f0 e8 12 01 1d 00 85 c0 74 ee eb ad 8d b6 00 00 00 00 >8d bf > > > >>>>>>>>>> >>>>>>>>>> So I am very confused: >>>>>>>>>> >>>>>>>>>> First, what's the difference between these two patches? Or say, does >>>>>>>>>> kvm support x2apic since kernel 2.6.32, or since kernel 3.9? >>>>>>>>>> >>>>>>>>> kvm support x2apic since kernel 2.6.32, but not support virtual x2apic mode until kernel 3.9. >>>>>>>>> >>>>>>>>>> Second, can guest use x2apic even if the host does not? (Assuming qemu >>>>>>>>>>has exposed this feature to guest.) The word "use" means something >>>>>>>>>> like, accessing x2apic registers, setting the x2apic enable bit in >>>>>>>>>> IA32_APICBASE MSR (i.e. bit 10). >>>>>>>>>> >>>>>>>>> Yes, you can use x2apic even if the PCPU does not support, and benefit from it, like performance bonus from MSR accesses instead of MMIO. >>>>>>>>> But, if I remember correctly, if your guest does not support interrupt-remmping, you only can use physical destination mode when using x2apic, >>>>>>>>> please see enable_IR_x2apic(). >>>>>>>>> >>>>>>>>>> -Jidong -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
