On Wed, Oct 4, 2017 at 4:42 PM, Radim Krčmář <rkrcmar@xxxxxxxxxx> wrote: > 2017-10-04 12:10+0200, Ladi Prosek: >> On Tue, Oct 3, 2017 at 9:53 PM, Radim Krčmář <rkrcmar@xxxxxxxxxx> wrote: >> > 2017-09-25 10:09+0200, Ladi Prosek: >> >> Entering SMM while running in guest mode wasn't working very well because several >> >> pieces of the vcpu state were left set up for nested operation. >> >> >> >> Some of the issues observed: >> >> >> >> * L1 was getting unexpected VM exits (using L1 interception controls but running >> >> in SMM execution environment) >> >> * MMU was confused (walk_mmu was still set to nested_mmu) >> >> * INTERCEPT_SMI was not emulated for L1 (KVM never injected SVM_EXIT_SMI) >> >> >> >> Intel SDM actually prescribes the logical processor to "leave VMX operation" upon >> >> entering SMM in 34.14.1 Default Treatment of SMI Delivery. AMD doesn't seem to >> >> document this but they provide fields in the SMM state-save area to stash the >> >> current state of SVM. What we need to do is basically get out of guest mode for >> >> the duration of SMM. All this completely transparent to L1, i.e. L1 is not given >> >> control and no L1 observable state changes. >> > >> > The best description I found is in APM vol. 2, 15.22 SMM Support: >> > >> > • The simplest solution is to not intercept SMI signals. SMIs >> > encountered while in a guest context are taken from within the guest >> > context. In this case, the SMM handler is not subject to any >> > intercepts set up by the VMM and consequently runs outside of the >> > virtualization controls. The state saved in the SMM State-Save area as >> > seen by the SMM handler reflects the state of the guest that had been >> > running at the time the SMI was encountered. When the SMM handler >> > executes the RSM instruction, the processor returns to executing in >> > the guest context, and any modifications to the SMM State-Save area >> > made by the SMM handler are reflected in the guest state. >> > >> > I think that the last sentence is not implemented correctly: >> > svm_prep_enter_smm() loads the host state into VMCB and >> > enter_smm_save_state_64() then puts host state where the SMM handler >> > would expect guest state. >> >> That's true. It seems to be easier to switch to the nested guest after >> we've restored all the basics from the SMM area. I briefly tried to >> reverse the order, which should make it compliant with 15.22 quoted >> above and hopefully also with what Intel wants (it's not clear to me >> whether their "leave VMX operation" in 34.14.1 Default Treatment of >> SMI Delivery means switching to host state or not). I quickly ran into >> issues though because entering the nested guest doesn't quite work if >> the vCPU is still in real mode. > > We basically want to keep the L2 state while removing L2 controls. > I think the current order could be maintained if we copied L2 state into > L1 before running current SMM routines. Afterward, we'd copy restored > L1 state into L2. > > It is inefficient, but should be simple. :) I've been able to make it work like this: - On SMM entry we save the L2 state into the state-save area. - Then we exit to L1 and continue with SMM entry as usual. - On RSM we restore the L2 state from the state-save area (while still in L1, i.e. is_guest_mode == false). - Then we re-enter L2. - Finally we restore the L2 state again (with a minor tweak as we're no longer switching from real to protected). If we didn't enter from L2, the second restore is omitted. Definitely inefficient but can be done reusing the existing routines. I'll send a v4 shortly. >> I think that the bootstrap sequence in em_rsm() has to be tweaked to >> make it possible to call post_leave_smm() before restoring state from >> the SMM area. I'll look into it. >> >> > Do Windows intercept SMI? >> >> Yes, but only if started with >1 vCPU. > > Interesting, thanks. I lied, Windows intercepts SMI always. It's just that on single CPU I was able to get away without emulating the interception for some reason. Which means that all this restore-after-RSM logic won't be exercised with Windows guests on AMD. It's still going to be used on Intel though and the code is common for both. Thanks!
