On 26/10/2017 13:48, Wanpeng Li wrote: > 2017-10-26 14:52 GMT+08:00 Nadav Amit <nadav.amit@xxxxxxxxx>: >> Wanpeng Li <kernellwp@xxxxxxxxx> wrote: >> >>> 2017-10-26 0:20 GMT+08:00 Nadav Amit <nadav.amit@xxxxxxxxx>: >>>> Wanpeng Li <kernellwp@xxxxxxxxx> wrote: >>>> >>>>> Cc Radim, Nadav, >>>>> 2017-10-24 19:10 GMT+08:00 Pedro Fonseca <pfonseca@xxxxxxxxxxxxxxxxx>: >>>>>> Hi, >>>>>> >>>>>> During tests that we conducted on KVM, we noticed that executing a "PUSH >>>>>> %ES" instruction under KVM produces different results on both memory and the >>>>>> SP register depending on whether EPT support is enabled. With EPT the SP is >>>>>> reduced by 4 bytes (and the written value is 0-padded) but without EPT >>>>>> support it is only reduced by 2 bytes. The difference can be observed when >>>>>> the CS.DB field is 1 (32-bit) but not when it's 0 (16-bit). >>>>>> >>>>>> The test case initializes the VM with EIP=0, CS.DB=1, ES=0x10, and SP=0xFFE. >>>>>> Memory is initialized with 0x06 (PUSH %ES) and 0xF4 (HLT). The testing >>>>>> system was running Linux 4.12.5 and Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz. >>>>>> >>>>>> The test case (https://pastebin.com/ZejdtGEk) produces the output bellow. >>>>>> Note that 0x10 is written to 0xFFA on EPT=1 but it's written to 0xFFC on >>>>>> EPT=0. >>>>>>> $ insmod kvm-intel.ko >>>>>>> $ sudo ./reproduce-push_es >>>>>>> Executing KVM_RUN >>>>>>> KVM_RUN exited (exit_reason: 5, KVM_EXIT_HLT) >>>>>>> 0000: 06 f4 00 00 00 00 00 00 >>>>>>> 0008: 00 00 00 00 00 00 00 00 >>>>>>> 0ff8: 00 00 10 00 00 00 00 00 >>>>>>> 1000: 00 00 00 00 00 00 00 00 >>>>>> >>>>>> >>>>>>> $ insmod kvm-intel.ko ept=0 >>>>>>> $ sudo ./reproduce-push_es >>>>>>> Executing KVM_RUN >>>>>>> KVM_RUN exited (exit_reason: 5, KVM_EXIT_HLT) >>>>>>> 0000: 06 f4 00 00 00 00 00 00 >>>>>>> 0008: 00 00 00 00 00 00 00 00 >>>>>>> 0ff8: 00 00 00 00 10 00 00 00 >>>>>>> 1000: 00 00 00 00 00 00 00 00 >>>>> >>>>> The cause of your two reports are the same. I think it has associated >>>>> with EPT+unrestricted_guest and vm8086 instead of EPT itself. vm8086 >>>>> emulates a real mode environment, so it will not respect CS.D=1 which >>>>> you give since there is no segment descriptors support. However, big >>>>> real mode is different, they still load the segment descriptors which >>>>> hand over from protect mode before the mode switch. Your testcase just >>>>> start a real mode guest in all its life time w/o switch to protect >>>>> mode or vice versa. And KVM(EPT=Y, unrestricted_guest=Y) can't >>>>> distinguish between a real mode guest w/ segment descriptors given by >>>>> userspace and big real mode which occurs when protect mode switch to >>>>> real mode. >>>> >>>> Interesting. I can guess that the Intel tests that I was running back at the >>>> time had a setup code (prior to the random code) in protected-mode, which >>>> would explain why I missed this problem. >>>> >>>> Perhaps the problem comes from wrong setting of the guest segment selector >>>> “unusable” bit. I see there are quite few hacks in the code in regard to >>>> this bit. >>> >>> I change the "present" bit of CS/DS/SS/ES to 0 in the testcase, >>> however, the guest vmentry fails. In addition, is there any idea how >>> to fix it in kvm? I can be the volunteer to implement the idea. :) >> >> Stupid me. I didn’t read the setup well enough. So I understand there is >> actually emulation when EPT=0, and this emulation is wrong. >> >> I don’t see where the operand size (op_bytes) for “Stack” operations in >> x86_decode_insn() is updated in respect to cs.d, and there is also no >> appropriate logic in em_push_sreg(). > > Do you mean vm8086 should still respect cs.d even if there is no > segment descriptors in real mode? Yes, the segment descriptors exist everywhere---in real mode and, to a lesser extent, in vm8086 mode, they are hidden behind the descriptor cache, but they are there. Paolo
