On Tue, Feb 27, 2024 at 08:40:26AM +0100, Jiri Slaby wrote: > On 27. 02. 24, 6:00, Pawan Gupta wrote: > > commit baf8361e54550a48a7087b603313ad013cc13386 upstream. > > > > MDS mitigation requires clearing the CPU buffers before returning to > > user. This needs to be done late in the exit-to-user path. Current > > location of VERW leaves a possibility of kernel data ending up in CPU > > buffers for memory accesses done after VERW such as: > > > > 1. Kernel data accessed by an NMI between VERW and return-to-user can > > remain in CPU buffers since NMI returning to kernel does not > > execute VERW to clear CPU buffers. > > 2. Alyssa reported that after VERW is executed, > > CONFIG_GCC_PLUGIN_STACKLEAK=y scrubs the stack used by a system > > call. Memory accesses during stack scrubbing can move kernel stack > > contents into CPU buffers. > > 3. When caller saved registers are restored after a return from > > function executing VERW, the kernel stack accesses can remain in > > CPU buffers(since they occur after VERW). > > > > To fix this VERW needs to be moved very late in exit-to-user path. > > > > In preparation for moving VERW to entry/exit asm code, create macros > > that can be used in asm. Also make VERW patching depend on a new feature > > flag X86_FEATURE_CLEAR_CPU_BUF. > ... > > --- a/arch/x86/include/asm/nospec-branch.h > > +++ b/arch/x86/include/asm/nospec-branch.h > > @@ -315,6 +315,17 @@ > > #endif > > .endm > > +/* > > + * Macro to execute VERW instruction that mitigate transient data sampling > > + * attacks such as MDS. On affected systems a microcode update overloaded VERW > > + * instruction to also clear the CPU buffers. VERW clobbers CFLAGS.ZF. > > + * > > + * Note: Only the memory operand variant of VERW clears the CPU buffers. > > + */ > > +.macro CLEAR_CPU_BUFFERS > > + ALTERNATIVE "", __stringify(verw mds_verw_sel), X86_FEATURE_CLEAR_CPU_BUF > > Why is not rip-relative preserved here? Because Nikolay reported that it was creating a problem for backports on kernels that don't support relocations in alternatives. More on this here: https://lore.kernel.org/lkml/20558f89-299b-472e-9a96-171403a83bd6@xxxxxxxx/ Also, RIP-relative addressing was a requirement only for the initial versions of the series, where the VERW operand was pointing within the macro. For performance gains, later versions switched to the implementation in which all VERW sites were pointing to single memory location. With that, RIP-relative addressing could be droped in favor of fixed addresses. > Will this work at all (it looks like verw would now touch random > memory)? AFAIK, all memory operand variants of VERW have the CPU buffer clearing behavior. I will confirm this with the CPU architects. > In any way, should you do any changes during the backport, you shall > document that. Sorry, I missed to mention this change in 6.7.y backport. I did include this info in the other backports I sent: https://lore.kernel.org/stable/20240226-delay-verw-backport-6-6-y-v1-0-aa17b2922725@xxxxxxxxxxxxxxx/T/ https://lore.kernel.org/stable/20240226-delay-verw-backport-6-1-y-v1-0-b3a2c5b9b0cb@xxxxxxxxxxxxxxx/T/
