This is a note to let you know that I've just added the patch titled x86/speculation: Fill RSB on vmexit for IBRS to the 5.15-stable tree which can be found at: http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary The filename of the patch is: x86-speculation-fill-rsb-on-vmexit-for-ibrs.patch and it can be found in the queue-5.15 subdirectory. If you, or anyone else, feels it should not be added to the stable tree, please let <stable@xxxxxxxxxxxxxxx> know about it. >From foo@baz Tue Jul 12 05:06:57 PM CEST 2022 From: Josh Poimboeuf <jpoimboe@xxxxxxxxxx> Date: Tue, 14 Jun 2022 23:16:15 +0200 Subject: x86/speculation: Fill RSB on vmexit for IBRS From: Josh Poimboeuf <jpoimboe@xxxxxxxxxx> commit 9756bba28470722dacb79ffce554336dd1f6a6cd upstream. Prevent RSB underflow/poisoning attacks with RSB. While at it, add a bunch of comments to attempt to document the current state of tribal knowledge about RSB attacks and what exactly is being mitigated. Signed-off-by: Josh Poimboeuf <jpoimboe@xxxxxxxxxx> Signed-off-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx> Signed-off-by: Borislav Petkov <bp@xxxxxxx> Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@xxxxxxxxxxxxx> Signed-off-by: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx> --- arch/x86/include/asm/cpufeatures.h | 2 - arch/x86/kernel/cpu/bugs.c | 63 ++++++++++++++++++++++++++++++++++--- arch/x86/kvm/vmx/vmenter.S | 6 +-- 3 files changed, 62 insertions(+), 9 deletions(-) --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -204,7 +204,7 @@ /* FREE! ( 7*32+10) */ #define X86_FEATURE_PTI ( 7*32+11) /* Kernel Page Table Isolation enabled */ #define X86_FEATURE_KERNEL_IBRS ( 7*32+12) /* "" Set/clear IBRS on kernel entry/exit */ -/* FREE! ( 7*32+13) */ +#define X86_FEATURE_RSB_VMEXIT ( 7*32+13) /* "" Fill RSB on VM-Exit */ #define X86_FEATURE_INTEL_PPIN ( 7*32+14) /* Intel Processor Inventory Number */ #define X86_FEATURE_CDP_L2 ( 7*32+15) /* Code and Data Prioritization L2 */ #define X86_FEATURE_MSR_SPEC_CTRL ( 7*32+16) /* "" MSR SPEC_CTRL is implemented */ --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -1394,17 +1394,70 @@ static void __init spectre_v2_select_mit pr_info("%s\n", spectre_v2_strings[mode]); /* - * If spectre v2 protection has been enabled, unconditionally fill - * RSB during a context switch; this protects against two independent - * issues: + * If Spectre v2 protection has been enabled, fill the RSB during a + * context switch. In general there are two types of RSB attacks + * across context switches, for which the CALLs/RETs may be unbalanced. * - * - RSB underflow (and switch to BTB) on Skylake+ - * - SpectreRSB variant of spectre v2 on X86_BUG_SPECTRE_V2 CPUs + * 1) RSB underflow + * + * Some Intel parts have "bottomless RSB". When the RSB is empty, + * speculated return targets may come from the branch predictor, + * which could have a user-poisoned BTB or BHB entry. + * + * AMD has it even worse: *all* returns are speculated from the BTB, + * regardless of the state of the RSB. + * + * When IBRS or eIBRS is enabled, the "user -> kernel" attack + * scenario is mitigated by the IBRS branch prediction isolation + * properties, so the RSB buffer filling wouldn't be necessary to + * protect against this type of attack. + * + * The "user -> user" attack scenario is mitigated by RSB filling. + * + * 2) Poisoned RSB entry + * + * If the 'next' in-kernel return stack is shorter than 'prev', + * 'next' could be tricked into speculating with a user-poisoned RSB + * entry. + * + * The "user -> kernel" attack scenario is mitigated by SMEP and + * eIBRS. + * + * The "user -> user" scenario, also known as SpectreBHB, requires + * RSB clearing. + * + * So to mitigate all cases, unconditionally fill RSB on context + * switches. + * + * FIXME: Is this pointless for retbleed-affected AMD? */ setup_force_cpu_cap(X86_FEATURE_RSB_CTXSW); pr_info("Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch\n"); /* + * Similar to context switches, there are two types of RSB attacks + * after vmexit: + * + * 1) RSB underflow + * + * 2) Poisoned RSB entry + * + * When retpoline is enabled, both are mitigated by filling/clearing + * the RSB. + * + * When IBRS is enabled, while #1 would be mitigated by the IBRS branch + * prediction isolation protections, RSB still needs to be cleared + * because of #2. Note that SMEP provides no protection here, unlike + * user-space-poisoned RSB entries. + * + * eIBRS, on the other hand, has RSB-poisoning protections, so it + * doesn't need RSB clearing after vmexit. + */ + if (boot_cpu_has(X86_FEATURE_RETPOLINE) || + boot_cpu_has(X86_FEATURE_KERNEL_IBRS)) + setup_force_cpu_cap(X86_FEATURE_RSB_VMEXIT); + + /* * Retpoline protects the kernel, but doesn't protect firmware. IBRS * and Enhanced IBRS protect firmware too, so enable IBRS around * firmware calls only when IBRS / Enhanced IBRS aren't otherwise --- a/arch/x86/kvm/vmx/vmenter.S +++ b/arch/x86/kvm/vmx/vmenter.S @@ -193,15 +193,15 @@ SYM_INNER_LABEL(vmx_vmexit, SYM_L_GLOBAL * IMPORTANT: RSB filling and SPEC_CTRL handling must be done before * the first unbalanced RET after vmexit! * - * For retpoline, RSB filling is needed to prevent poisoned RSB entries - * and (in some cases) RSB underflow. + * For retpoline or IBRS, RSB filling is needed to prevent poisoned RSB + * entries and (in some cases) RSB underflow. * * eIBRS has its own protection against poisoned RSB, so it doesn't * need the RSB filling sequence. But it does need to be enabled * before the first unbalanced RET. */ - FILL_RETURN_BUFFER %_ASM_CX, RSB_CLEAR_LOOPS, X86_FEATURE_RETPOLINE + FILL_RETURN_BUFFER %_ASM_CX, RSB_CLEAR_LOOPS, X86_FEATURE_RSB_VMEXIT pop %_ASM_ARG2 /* @flags */ pop %_ASM_ARG1 /* @vmx */ Patches currently in stable-queue which might be from jpoimboe@xxxxxxxxxx are queue-5.15/x86-sev-avoid-using-__x86_return_thunk.patch queue-5.15/x86-ftrace-use-alternative-ret-encoding.patch queue-5.15/objtool-re-add-unwind_hint_-save_restore.patch queue-5.15/x86-bugs-add-retbleed-ibpb.patch queue-5.15/x86-retpoline-cleanup-some-ifdefery.patch queue-5.15/kvm-vmx-flatten-__vmx_vcpu_run.patch queue-5.15/x86-cpu-amd-add-spectral-chicken.patch queue-5.15/kvm-vmx-fix-ibrs-handling-after-vmexit.patch queue-5.15/kvm-vmx-prevent-guest-rsb-poisoning-attacks-with-eibrs.patch queue-5.15/x86-vsyscall_emu-64-don-t-use-ret-in-vsyscall-emulation.patch queue-5.15/objtool-skip-non-text-sections-when-adding-return-thunk-sites.patch queue-5.15/x86-bugs-do-ibpb-fallback-check-only-once.patch queue-5.15/x86-add-magic-amd-return-thunk.patch queue-5.15/x86-bugs-keep-a-per-cpu-ia32_spec_ctrl-value.patch queue-5.15/x86-objtool-create-.return_sites.patch queue-5.15/x86-kvm-fix-setcc-emulation-for-return-thunks.patch queue-5.15/x86-retpoline-swizzle-retpoline-thunk.patch queue-5.15/x86-speculation-fix-firmware-entry-spec_ctrl-handling.patch queue-5.15/x86-speculation-add-spectre_v2-ibrs-option-to-support-kernel-ibrs.patch queue-5.15/x86-xen-add-untrain_ret.patch queue-5.15/x86-undo-return-thunk-damage.patch queue-5.15/x86-speculation-remove-x86_spec_ctrl_mask.patch queue-5.15/x86-entry-avoid-very-early-ret.patch queue-5.15/x86-speculation-fill-rsb-on-vmexit-for-ibrs.patch queue-5.15/objtool-add-entry-unret-validation.patch queue-5.15/kvm-vmx-convert-launched-argument-to-flags.patch queue-5.15/x86-bpf-use-alternative-ret-encoding.patch queue-5.15/x86-bugs-split-spectre_v2_select_mitigation-and-spectre_v2_user_select_mitigation.patch queue-5.15/x86-bugs-report-intel-retbleed-vulnerability.patch queue-5.15/x86-cpufeatures-move-retpoline-flags-to-word-11.patch queue-5.15/x86-speculation-fix-spec_ctrl-write-on-smt-state-change.patch queue-5.15/x86-retpoline-use-mfunction-return.patch queue-5.15/x86-xen-rename-sys-entry-points.patch queue-5.15/x86-bugs-optimize-spec_ctrl-msr-writes.patch queue-5.15/x86-bugs-report-amd-retbleed-vulnerability.patch queue-5.15/x86-static_call-use-alternative-ret-encoding.patch queue-5.15/x86-speculation-fix-rsb-filling-with-config_retpoline-n.patch queue-5.15/x86-use-return-thunk-in-asm-code.patch queue-5.15/intel_idle-disable-ibrs-during-long-idle.patch queue-5.15/x86-speculation-use-cached-host-spec_ctrl-value-for-guest-entry-exit.patch queue-5.15/x86-bugs-add-amd-retbleed-boot-parameter.patch queue-5.15/x86-entry-add-kernel-ibrs-implementation.patch queue-5.15/objtool-treat-.text.__x86.-as-noinstr.patch queue-5.15/objtool-update-retpoline-validation.patch