This is a note to let you know that I've just added the patch titled
x86/cpu/kvm: Provide UNTRAIN_RET_VM
to the 6.4-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary
The filename of the patch is:
x86-cpu-kvm-provide-untrain_ret_vm.patch
and it can be found in the queue-6.4 subdirectory.
If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@xxxxxxxxxxxxxxx> know about it.
>From 864bcaa38ee44ec6c0e43f79c2d2997b977e26b2 Mon Sep 17 00:00:00 2001
From: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Date: Mon, 14 Aug 2023 13:44:35 +0200
Subject: x86/cpu/kvm: Provide UNTRAIN_RET_VM
From: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
commit 864bcaa38ee44ec6c0e43f79c2d2997b977e26b2 upstream.
Similar to how it doesn't make sense to have UNTRAIN_RET have two
untrain calls, it also doesn't make sense for VMEXIT to have an extra
IBPB call.
This cures VMEXIT doing potentially unret+IBPB or double IBPB.
Also, the (SEV) VMEXIT case seems to have been overlooked.
Redefine the meaning of the synthetic IBPB flags to:
- ENTRY_IBPB -- issue IBPB on entry (was: entry + VMEXIT)
- IBPB_ON_VMEXIT -- issue IBPB on VMEXIT
And have 'retbleed=ibpb' set *BOTH* feature flags to ensure it retains
the previous behaviour and issues IBPB on entry+VMEXIT.
The new 'srso=ibpb_vmexit' option only sets IBPB_ON_VMEXIT.
Create UNTRAIN_RET_VM specifically for the VMEXIT case, and have that
check IBPB_ON_VMEXIT.
All this avoids having the VMEXIT case having to check both ENTRY_IBPB
and IBPB_ON_VMEXIT and simplifies the alternatives.
Fixes: fb3bd914b3ec ("x86/srso: Add a Speculative RAS Overflow mitigation")
Signed-off-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>
Signed-off-by: Borislav Petkov (AMD) <bp@xxxxxxxxx>
Link: https://lore.kernel.org/r/20230814121149.109557833@xxxxxxxxxxxxx
Signed-off-by: Greg Kroah-Hartman <gregkh@xxxxxxxxxxxxxxxxxxx>
---
arch/x86/include/asm/nospec-branch.h | 11 +++++++++++
arch/x86/kernel/cpu/bugs.c | 1 +
arch/x86/kvm/svm/vmenter.S | 7 ++-----
3 files changed, 14 insertions(+), 5 deletions(-)
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -295,6 +295,17 @@
#endif
.endm
+.macro UNTRAIN_RET_VM
+#if defined(CONFIG_CPU_UNRET_ENTRY) || defined(CONFIG_CPU_IBPB_ENTRY) || \
+ defined(CONFIG_CALL_DEPTH_TRACKING) || defined(CONFIG_CPU_SRSO)
+ VALIDATE_UNRET_END
+ ALTERNATIVE_3 "", \
+ CALL_UNTRAIN_RET, X86_FEATURE_UNRET, \
+ "call entry_ibpb", X86_FEATURE_IBPB_ON_VMEXIT, \
+ __stringify(RESET_CALL_DEPTH), X86_FEATURE_CALL_DEPTH
+#endif
+.endm
+
.macro UNTRAIN_RET_FROM_CALL
#if defined(CONFIG_CPU_UNRET_ENTRY) || defined(CONFIG_CPU_IBPB_ENTRY) || \
defined(CONFIG_CALL_DEPTH_TRACKING)
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -1054,6 +1054,7 @@ do_cmd_auto:
case RETBLEED_MITIGATION_IBPB:
setup_force_cpu_cap(X86_FEATURE_ENTRY_IBPB);
+ setup_force_cpu_cap(X86_FEATURE_IBPB_ON_VMEXIT);
mitigate_smt = true;
break;
--- a/arch/x86/kvm/svm/vmenter.S
+++ b/arch/x86/kvm/svm/vmenter.S
@@ -222,10 +222,7 @@ SYM_FUNC_START(__svm_vcpu_run)
* because interrupt handlers won't sanitize 'ret' if the return is
* from the kernel.
*/
- UNTRAIN_RET
-
- /* SRSO */
- ALTERNATIVE "", "call entry_ibpb", X86_FEATURE_IBPB_ON_VMEXIT
+ UNTRAIN_RET_VM
/*
* Clear all general purpose registers except RSP and RAX to prevent
@@ -362,7 +359,7 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run)
* because interrupt handlers won't sanitize RET if the return is
* from the kernel.
*/
- UNTRAIN_RET
+ UNTRAIN_RET_VM
/* "Pop" @spec_ctrl_intercepted. */
pop %_ASM_BX
Patches currently in stable-queue which might be from peterz@xxxxxxxxxxxxx are
queue-6.4/x86-cpu-fix-up-srso_safe_ret-and-__x86_return_thunk.patch
queue-6.4/x86-static_call-fix-__static_call_fixup.patch
queue-6.4/x86-cpu-cleanup-the-untrain-mess.patch
queue-6.4/iopoll-call-cpu_relax-in-busy-loops.patch
queue-6.4/objtool-x86-fixup-frame-pointer-vs-rethunk.patch
queue-6.4/x86-retpoline-kprobes-skip-optprobe-check-for-indirect-jumps-with-retpolines-and-ibt.patch
queue-6.4/x86-retpoline-kprobes-fix-position-of-thunk-sections-with-config_lto_clang.patch
queue-6.4/x86-cpu-kvm-provide-untrain_ret_vm.patch
queue-6.4/x86-cpu-rename-srso_-.-_alias-to-srso_alias_-1.patch
queue-6.4/x86-alternative-make-custom-return-thunk-unconditional.patch
queue-6.4/x86-cpu-rename-original-retbleed-methods.patch
queue-6.4/objtool-x86-fix-srso-mess.patch
queue-6.4/x86-cpu-clean-up-srso-return-thunk-mess.patch
queue-6.4/x86-cpu-fix-__x86_return_thunk-symbol-type.patch
