Wanpeng Li <kernellwp@xxxxxxxxx> wrote: > 2017-10-26 0:20 GMT+08:00 Nadav Amit <nadav.amit@xxxxxxxxx>: >> Wanpeng Li <kernellwp@xxxxxxxxx> wrote: >> >>> Cc Radim, Nadav, >>> 2017-10-24 19:10 GMT+08:00 Pedro Fonseca <pfonseca@xxxxxxxxxxxxxxxxx>: >>>> Hi, >>>> >>>> During tests that we conducted on KVM, we noticed that executing a "PUSH >>>> %ES" instruction under KVM produces different results on both memory and the >>>> SP register depending on whether EPT support is enabled. With EPT the SP is >>>> reduced by 4 bytes (and the written value is 0-padded) but without EPT >>>> support it is only reduced by 2 bytes. The difference can be observed when >>>> the CS.DB field is 1 (32-bit) but not when it's 0 (16-bit). >>>> >>>> The test case initializes the VM with EIP=0, CS.DB=1, ES=0x10, and SP=0xFFE. >>>> Memory is initialized with 0x06 (PUSH %ES) and 0xF4 (HLT). The testing >>>> system was running Linux 4.12.5 and Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz. >>>> >>>> The test case (https://pastebin.com/ZejdtGEk) produces the output bellow. >>>> Note that 0x10 is written to 0xFFA on EPT=1 but it's written to 0xFFC on >>>> EPT=0. >>>>> $ insmod kvm-intel.ko >>>>> $ sudo ./reproduce-push_es >>>>> Executing KVM_RUN >>>>> KVM_RUN exited (exit_reason: 5, KVM_EXIT_HLT) >>>>> 0000: 06 f4 00 00 00 00 00 00 >>>>> 0008: 00 00 00 00 00 00 00 00 >>>>> 0ff8: 00 00 10 00 00 00 00 00 >>>>> 1000: 00 00 00 00 00 00 00 00 >>>> >>>> >>>>> $ insmod kvm-intel.ko ept=0 >>>>> $ sudo ./reproduce-push_es >>>>> Executing KVM_RUN >>>>> KVM_RUN exited (exit_reason: 5, KVM_EXIT_HLT) >>>>> 0000: 06 f4 00 00 00 00 00 00 >>>>> 0008: 00 00 00 00 00 00 00 00 >>>>> 0ff8: 00 00 00 00 10 00 00 00 >>>>> 1000: 00 00 00 00 00 00 00 00 >>> >>> The cause of your two reports are the same. I think it has associated >>> with EPT+unrestricted_guest and vm8086 instead of EPT itself. vm8086 >>> emulates a real mode environment, so it will not respect CS.D=1 which >>> you give since there is no segment descriptors support. However, big >>> real mode is different, they still load the segment descriptors which >>> hand over from protect mode before the mode switch. Your testcase just >>> start a real mode guest in all its life time w/o switch to protect >>> mode or vice versa. And KVM(EPT=Y, unrestricted_guest=Y) can't >>> distinguish between a real mode guest w/ segment descriptors given by >>> userspace and big real mode which occurs when protect mode switch to >>> real mode. >> >> Interesting. I can guess that the Intel tests that I was running back at the >> time had a setup code (prior to the random code) in protected-mode, which >> would explain why I missed this problem. >> >> Perhaps the problem comes from wrong setting of the guest segment selector >> “unusable” bit. I see there are quite few hacks in the code in regard to >> this bit. > > I change the "present" bit of CS/DS/SS/ES to 0 in the testcase, > however, the guest vmentry fails. In addition, is there any idea how > to fix it in kvm? I can be the volunteer to implement the idea. :) Stupid me. I didn’t read the setup well enough. So I understand there is actually emulation when EPT=0, and this emulation is wrong. I don’t see where the operand size (op_bytes) for “Stack” operations in x86_decode_insn() is updated in respect to cs.d, and there is also no appropriate logic in em_push_sreg(). I am sorry, but don’t have the time to further look into it right now. Regards, Nadav
