On Fri, Jan 29, 2021 at 03:38:52PM +0100, Paolo Bonzini wrote: > On 29/01/21 13:17, Yang Weijiang wrote: > > > > It's specific to VM case, during VM reboot, memory mode reset but VM_ENTRY_LOAD_CET_STATE > > > > is still set, and VMCS contains stale GUEST_SSP, this hits vm-entry failure > > > > documented in 10.7 VM Entry at: > > > > https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf > > > > Since CR4.CET is also reset during VM reboot, to take the change to clear the stale data. > > > > Maybe I need to find a better place to do the things. > > > Then you must use a field of struct vmx_vcpu instead of the VMCS to hold > > > GUEST_SSP (while GUEST_S_CET and GUEST_INTR_SSP_TABLE should not be an > > > issue). > > > > > Sorry, I don't get your point, can I just clear the GUEST_SSP field in this case? > > Anyway save/restore GUEST_SSP via VMCS is an efficient way. > > You cannot clear it, because it is preserved when CR4.CET is modified. > > However, I checked the latest SDM and the GUEST_SSP rules are changed to > just this: > > SSP. The following checks are performed if the “load CET state” VM-entry > control is 1 > — Bits 1:0 must be 0. > — If the processor supports the Intel 64 architecture, bits 63:N must be > identical, where N is the CPU’s maximum linear-address width. (This check > does not apply if the processor supports 64 linear-address bits.) The guest > SSP value is not required to be canonical; the value of bit N-1 may differ > from that of bit N. > > In particular it doesn't mention the "IA-32e mode guest" VM-entry control or > the CS.L bit anymore, so it should not be necessary anymore to even reset > SSP to 0, and you can keep GUEST_SSP in the VMCS. > There could be some gaps between the two specs, I tested VM reboot with these patches, if I don't clear GUEST_SSP field while CR4.CET is changed, then it always encounters the vm-entry failure issue, the VMCS dump is like below: [341485.265277] *** Guest State *** [341485.265280] CR0: actual=0x0000000000000030, shadow=0x0000000060000010, gh_mask=fffffffffffffff7 [341485.265281] CR4: actual=0x0000000000002040, shadow=0x0000000000000000, gh_mask=fffffffffffef871 [341485.265282] CR3 = 0x0000000000000000 [341485.265283] RSP = 0x0000000000000000 RIP = 0x000000000000fff0 [341485.265283] RFLAGS=0x00000002 DR7 = 0x0000000000000400 [341485.265284] Sysenter RSP=0000000000000000 CS:RIP=0000:0000000000000000 [341485.265285] CS: sel=0xf000, attr=0x0009b, limit=0x0000ffff, base=0x00000000ffff0000 [341485.265286] DS: sel=0x0000, attr=0x00093, limit=0x0000ffff, base=0x0000000000000000 [341485.265287] SS: sel=0x0000, attr=0x00093, limit=0x0000ffff, base=0x0000000000000000 [341485.265288] ES: sel=0x0000, attr=0x00093, limit=0x0000ffff, base=0x0000000000000000 [341485.265288] FS: sel=0x0000, attr=0x00093, limit=0x0000ffff, base=0x0000000000000000 [341485.265289] GS: sel=0x0000, attr=0x00093, limit=0x0000ffff, base=0x0000000000000000 [341485.265289] GDTR: limit=0x0000ffff, base=0x0000000000000000 [341485.265290] LDTR: sel=0x0000, attr=0x00082, limit=0x0000ffff, base=0x0000000000000000 [341485.265291] IDTR: limit=0x0000ffff, base=0x0000000000000000 [341485.265291] TR: sel=0x0000, attr=0x0008b, limit=0x0000ffff, base=0x0000000000000000 [341485.265292] EFER = 0x0000000000000000 PAT = 0x0007040600070406 [341485.265292] DebugCtl = 0x0000000000000000 DebugExceptions = 0x0000000000000000 [341485.265293] Interruptibility = 00000000 ActivityState = 00000000 [341485.265294] InterruptStatus = 0000 [341485.265294] S_CET = 0x0000000000000000 [341485.265295] SSP = 0x00007fc1727fdfd0 [341485.265295] SSP TABLE = 0x0000000000000000 [341485.265296] *** Host State *** [341485.265296] RIP = 0xffffffffc1235900 RSP = 0xffffaed7c150bd68 [341485.265297] CS=0010 SS=0018 DS=0000 ES=0000 FS=0000 GS=0000 TR=0040 [341485.265298] FSBase=00007f6851d36700 GSBase=ffff9de2e0980000 TRBase=fffffe0000141000 [341485.265298] GDTBase=fffffe000013f000 IDTBase=fffffe0000000000 [341485.265299] CR0=0000000080050033 CR3=00000001135ae001 CR4=0000000000f72ee0 [341485.265300] Sysenter RSP=fffffe0000141000 CS:RIP=0010:ffffffff8dc015e0 [341485.265301] EFER = 0x0000000000000d01 PAT = 0x0407050600070106 [341485.265301] *** Control State *** [341485.265302] PinBased=000000ff CPUBased=b5a06dfa SecondaryExec=021237eb [341485.265303] EntryControls=0010d1ff ExitControls=102befff [341485.265303] ExceptionBitmap=00060042 PFECmask=00000000 PFECmatch=00000000 [341485.265304] VMEntry: intr_info=00000000 errcode=00000000 ilen=00000000 [341485.265305] VMExit: intr_info=00000000 errcode=00000000 ilen=00000002 [341485.265306] reason=80000021 qualification=0000000000000000 [341485.265306] IDTVectoring: info=00000000 errcode=00000000 [341485.265307] TSC Offset = 0xfffd10acb111a2a0 [341485.265307] TSC Multiplier = 0x0001000000000000 [341485.265308] SVI|RVI = 00|00 TPR Threshold = 0x00 [341485.265309] APIC-access addr = 0x0000000499d63000 virt-APIC addr = 0x000000011ade6000 [341485.265310] PostedIntrVec = 0xf2 [341485.265310] EPT pointer = 0x000000011fffc05e [341485.265310] PLE Gap=00000080 Window=00010000 [341485.265311] Virtual processor ID = 0x0001 [341485.265312] S_CET = 0x0000000000000000 [341485.265312] SSP = 0x0000000000000000 [341485.265312] SSP TABLE = 0x0000000000000000 The alternative way is to clear it in exit_lmode(), it also works. > Paolo
