The 'if_nospec' primitive marks locations where the kernel is disabling
speculative execution that could potentially access privileged data. It
is expected to be paired with a 'nospec_{ptr,load}' where the user
controlled value is actually consumed. Architectures can optionally
implement a speculation barrier in 'if_nospec' or a speculation sync in
each 'nospec_{ptr,load}' depending what is best/feasible for the
architecture.
The pairing of if_nospec and nospec_load documents which branch
is sensitive to which load(s) in a code block. For example:
if_nospec(foo < bar) {
ptr = array_base + foo;
array_max = array_base + bar;
baz = nospec_load(ptr, array_base, array_max);
}
Suggested-by: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Cc: Mark Rutland <mark.rutland@xxxxxxx>
Cc: Greg KH <gregkh@xxxxxxxxxxxxxxxxxxx>
Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
Cc: Alan Cox <alan@xxxxxxxxxxxxxxx>
Cc: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>
Cc: Elena Reshetova <elena.reshetova@xxxxxxxxx>
Signed-off-by: Dan Williams <dan.j.williams@xxxxxxxxx>
---
This proposal builds on Mark's nospec_load RFC here:
https://lkml.org/lkml/2018/1/3/754
...and combines it with a suggestion from Peter to introduce if_nospec.
The goal is to both document which loads must not be speculated behind
which branches, and allow architectures like x86 that can do a single
barrier after the branch to coexist with architectures like ARM that
want to instrument each load.
include/asm-generic/barrier.h | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)
diff --git a/include/asm-generic/barrier.h b/include/asm-generic/barrier.h
index 5eba6ae0c34e..25c1b47f84b7 100644
--- a/include/asm-generic/barrier.h
+++ b/include/asm-generic/barrier.h
@@ -55,6 +55,27 @@
#endif
/**
+ * if_nospec() - block speculative execution of this branch
+ *
+ * @cond: condition that if true should barrier speculation
+ *
+ * Architectures should override the definition of nospec_barrier() to
+ * inject a speculative execution barrier for any conditional branches
+ * that might speculatively execute reads based on user controlled
+ * value. Architectures that can more efficiently flush speculation via
+ * nospec_load can leave nospec_barrier as a nop.
+ *
+ * The expectation is that nospec_{load,ptr} are always used inside an
+ * if_nospec block so that architectures that can use a single barrier
+ * after the branch can do that once rather than per access.
+ */
+#define if_nospec(cond) if (({ bool ret = (cond); nospec_barrier(); ret }))
+
+#ifndef nospec_barrier
+#define nospec_barrier() do { } while (0)
+#endif
+
+/**
* nospec_ptr() - Ensure a pointer is bounded, even under speculation.
*
* @ptr: the pointer to test
@@ -68,6 +89,11 @@
* interval both under architectural execution and under speculation,
* preventing propagation of an out-of-bounds pointer to code which is
* speculatively executed.
+ *
+ * Architectures that can more efficiently flush speculation via
+ * nospec_barrier can define nospec_ptr as a nop.
+ *
+ * The expectation is that if_nospec and nospec_ptr are always paired.
*/
#ifndef nospec_ptr
#define nospec_ptr(ptr, lo, hi) \
@@ -93,6 +119,11 @@
* Architectures should override this to ensure that ptr falls in the [lo, hi)
* interval both under architectural execution and under speculation,
* preventing speculative out-of-bounds reads.
+ *
+ * Architectures that can more efficiently flush speculation via
+ * nospec_barrier can define nospec_load as a nop.
+ *
+ * The expectation is that if_nospec and nospec_load are always paired.
*/
#ifndef nospec_load
#define nospec_load(ptr, lo, hi) \
