Hi Paolo Do you mean the OVEM already transfer control to OS Waking vector? System stuck just because the page table in 0x9C000 is corrupt? Thank you Yao Jiewen -----Original Message----- From: Paolo Bonzini [mailto:pbonzini@xxxxxxxxxx] Sent: Friday, December 06, 2013 8:03 PM To: edk2-devel@xxxxxxxxxxxxxxxxxxxxx Cc: KVM devel mailing list Subject: Re: [edk2] apparent KVM problem with LRET in TianoCore S3 resume trampoline Il 05/12/2013 19:29, Laszlo Ersek ha scritto: > On 12/05/13 18:42, Paolo Bonzini wrote: >> Il 05/12/2013 17:12, Laszlo Ersek ha scritto: >>> Hi, >>> >>> I'm working on S3 suspend/resume in OVMF. The problem is that I'm >>> getting an unexpected guest reboot for code (LRET) that works on >>> physical hardware. I tried to trace the problem with ftrace, but I >>> didn't get any mentions of em_ret_far(). (Maybe I was looking in the >>> wrong place.) >> >> What does ftrace say anyway? > > (pls. see in the next msg I sent) Actually I meant the ftrace without any patches. Thanks to your binary I now reproduced the issue and it looks like the 64-bit->16-bit switch works: qemu-system-x86-4081 [001] 62650.335040: kvm_exit: reason CR_ACCESS rip 0x3cf7ae45 info 0 0 qemu-system-x86-4081 [001] 62650.335041: kvm_cr: cr_write 0 = 0x32 qemu-system-x86-4081 [001] 62650.335046: kvm_entry: vcpu 0 This is the "mov %rax, %cr0". PE and PG are turned off. qemu-system-x86-4081 [001] 62650.335047: kvm_exit: reason MSR_READ rip 0x3cf7ae4e info 0 0 qemu-system-x86-4081 [001] 62650.335048: kvm_msr: msr_read c0000080 = 0x100 qemu-system-x86-4081 [001] 62650.335048: kvm_entry: vcpu 0 qemu-system-x86-4081 [001] 62650.335048: kvm_exit: reason MSR_WRITE rip 0x3cf7ae53 info 0 0 qemu-system-x86-4081 [001] 62650.335049: kvm_msr: msr_write c0000080 = 0x0 qemu-system-x86-4081 [001] 62650.335050: kvm_entry: vcpu 0 LME is turned off. qemu-system-x86-4081 [001] 62650.335050: kvm_exit: reason CR_ACCESS rip 0x3cf7ae55 info 304 0 qemu-system-x86-4081 [001] 62650.335050: kvm_cr: cr_write 4 = 0x640 qemu-system-x86-4081 [001] 62650.335053: kvm_entry: vcpu 0 PAE is turned off. qemu-system-x86-4081 [001] 62650.335054: kvm_exit: reason CR_ACCESS rip 0x11e6 info 0 0 qemu-system-x86-4081 [001] 62650.335054: kvm_cr: cr_write 0 = 0x33 qemu-system-x86-4081 [001] 62650.335054: kvm_entry: vcpu 0 Here we're already in real mode. The weird RIP is explained by the first few bytes after the FACS resume vector: 0x9a1d:0000: cli 0x9a1d:0001: cld 0x9a1d:0002: ljmp $9900,$11d7 9900:11d7 is the same physical address as 9a1d:0007. Fast forward a bit: qemu-system-x86-4081 [001] 62650.335071: kvm_exit: reason CR_ACCESS rip 0x9aec7 info 0 0 qemu-system-x86-4081 [001] 62650.335071: kvm_cr: cr_write 0 = 0x80010001 qemu-system-x86-4081 [001] 62650.335074: kvm_entry: vcpu 0 qemu-system-x86-4081 [001] 62650.335076: kvm_exit: reason TRIPLE_FAULT rip 0x0 info 0 0 Disassembling mixed 16-/32-/64-bit is a pain, so I ran QEMU with -no-shutdown -no-reboot and dumped the memory with (qemu) dump-guest-memory tramp.dmp 0x90000 0x10000 Lets look at the rest of the trace now. After the previous cr0 access we have: qemu-system-x86-4081 [001] 62650.335055: kvm_exit: reason CR_ACCESS rip 0x11fa info 0 0 qemu-system-x86-4081 [001] 62650.335055: kvm_cr: cr_write 0 = 0x32 qemu-system-x86-4081 [001] 62650.335055: kvm_entry: vcpu 0 It gets out to real mode again. No idea why. It does some setup. qemu-system-x86-4081 [001] 62650.335056: kvm_exit: reason MSR_WRITE rip 0x1258 info 0 0 qemu-system-x86-4081 [001] 62650.335056: kvm_msr: msr_write 1a0 = 0x1 qemu-system-x86-4081 [001] 62650.335057: kvm_entry: vcpu 0 qemu-system-x86-4081 [001] 62650.335057: kvm_exit: reason WBINVD rip 0x1001 info 0 0 qemu-system-x86-4081 [001] 62650.335057: kvm_entry: vcpu 0 qemu-system-x86-4081 [001] 62650.335058: kvm_exit: reason CPUID rip 0x1073 info 0 0 qemu-system-x86-4081 [001] 62650.335058: kvm_cpuid: func 0 rax 4 rbx 756e6547 rcx 6c65746e rdx 49656e69 qemu-system-x86-4081 [001] 62650.335059: kvm_entry: vcpu 0 qemu-system-x86-4081 [001] 62650.335059: kvm_exit: reason CPUID rip 0x10c0 info 0 0 qemu-system-x86-4081 [001] 62650.335059: kvm_cpuid: func 1 rax 663 rbx 800 rcx 80802001 rdx 78bfbfd qemu-system-x86-4081 [001] 62650.335059: kvm_entry: vcpu 0 qemu-system-x86-4081 [001] 62650.335060: kvm_exit: reason CPUID rip 0x10ff info 0 0 qemu-system-x86-4081 [001] 62650.335060: kvm_cpuid: func 1 rax 663 rbx 800 rcx 80802001 rdx 78bfbfd qemu-system-x86-4081 [001] 62650.335060: kvm_entry: vcpu 0 qemu-system-x86-4081 [001] 62650.335061: kvm_exit: reason CPUID rip 0x1117 info 0 0 qemu-system-x86-4081 [001] 62650.335061: kvm_cpuid: func 80000000 rax 8000000a rbx 756e6547 rcx 6c65746e rdx 49656e69 qemu-system-x86-4081 [001] 62650.335061: kvm_entry: vcpu 0 qemu-system-x86-4081 [001] 62650.335062: kvm_exit: reason CPUID rip 0x1127 info 0 0 qemu-system-x86-4081 [001] 62650.335062: kvm_cpuid: func 80000001 rax 663 rbx 0 rcx 1 rdx 2191abfd qemu-system-x86-4081 [001] 62650.335062: kvm_entry: vcpu 0 qemu-system-x86-4081 [001] 62650.335063: kvm_exit: reason CPUID rip 0x113f info 0 0 qemu-system-x86-4081 [001] 62650.335063: kvm_cpuid: func 1 rax 663 rbx 800 rcx 80802001 rdx 78bfbfd qemu-system-x86-4081 [001] 62650.335063: kvm_entry: vcpu 0 qemu-system-x86-4081 [001] 62650.335064: kvm_exit: reason CR_ACCESS rip 0x103c info 0 0 qemu-system-x86-4081 [001] 62650.335064: kvm_cr: cr_write 0 = 0x1 qemu-system-x86-4081 [001] 62650.335064: kvm_entry: vcpu 0 Enabling protected mode: 0009A036 66B801000000 mov eax,0x1 0009A03C 0F22C0 mov cr0,eax 0009A03F 66EA90AE09000800 jmp dword 0x8:0x9ae90 This is a 32-bit selector. 0009AE90 8ED2 mov ss,edx 0009AE92 81C400900900 add esp,0x99000 0009AE98 8EDA mov ds,edx 0009AE9A 8EC2 mov es,edx 0009AE9C 8EE2 mov fs,edx 0009AE9E 8EEA mov gs,edx qemu-system-x86-4081 [001] 62650.335065: kvm_exit: reason CR_ACCESS rip 0x9aea5 info 4 0 qemu-system-x86-4081 [001] 62650.335065: kvm_cr: cr_write 4 = 0x6f0 qemu-system-x86-4081 [001] 62650.335066: kvm_entry: vcpu 0 Enabling PAE: 0009AEA0 A110D00900 mov eax,[0x9d010] 0009AEA5 0F22E0 mov cr4,eax qemu-system-x86-4081 [001] 62650.335067: kvm_exit: reason CR_ACCESS rip 0x9aead info 3 0 qemu-system-x86-4081 [001] 62650.335067: kvm_cr: cr_write 3 = 0x9c000 qemu-system-x86-4081 [001] 62650.335068: kvm_entry: vcpu 0 Setting CR3 0009AEA8 B800C00900 mov eax,0x9c000 0009AEAD 0F22D8 mov cr3,eax qemu-system-x86-4081 [001] 62650.335068: kvm_exit: reason MSR_WRITE rip 0x9aec0 info 0 0 qemu-system-x86-4081 [001] 62650.335070: kvm_msr: msr_write c0000080 = 0x901 qemu-system-x86-4081 [001] 62650.335070: kvm_entry: vcpu 0 Enabling LME 0009AEB0 A108D00900 mov eax,[0x9d008] 0009AEB5 8B150CD00900 mov edx,[dword 0x9d00c] 0009AEBB B9800000C0 mov ecx,0xc0000080 0009AEC0 0F30 wrmsr qemu-system-x86-4081 [001] 62650.335071: kvm_exit: reason CR_ACCESS rip 0x9aec7 info 0 0 qemu-system-x86-4081 [001] 62650.335071: kvm_cr: cr_write 0 = 0x80010001 qemu-system-x86-4081 [001] 62650.335074: kvm_entry: vcpu 0 Enabling paging 0009AEC2 B801000180 mov eax,0x80010001 0009AEC7 0F22C0 mov cr0,eax But before we get here: 0009AECA EA30AF09001000 jmp dword 0x10:0x9af30 ... kaboom: qemu-system-x86-4081 [001] 62650.335076: kvm_exit: reason TRIPLE_FAULT rip 0x0 info 0 0 The page tables are, ahem, crap: 000c000: 6750 fe01 0000 0000 0000 0000 0000 0000 gP.............. 000c010: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000c020: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000c030: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000c040: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000c050: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000c060: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000c070: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000c080: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000c090: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000c0a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000c0b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000c0c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000c0d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000c0e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 000c0f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................ This is 0x9c000. Strikes any bell? Paolo ------------------------------------------------------------------------------ Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk _______________________________________________ edk2-devel mailing list edk2-devel@xxxxxxxxxxxxxxxxxxxxx https://lists.sourceforge.net/lists/listinfo/edk2-devel -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
