Add comments to the tdp_mmu_set_spte_atomic() and kvm_tdp_mmu_read_spte()
provide ordering guarantees to ensure that any changes made to a child
shadow page are guaranteed to be visible before a SPTE is marked present,
e.g. that there's no risk of concurrent readers observing a stale PFN for
a shadow-present SPTE.
No functional change intended.
Suggested-by: Paolo Bonzini <pbonzini@xxxxxxxxxx>
Signed-off-by: Sean Christopherson <seanjc@xxxxxxxxxx>
---
arch/x86/kvm/mmu/tdp_iter.h | 6 ++++++
arch/x86/kvm/mmu/tdp_mmu.c | 5 +++++
2 files changed, 11 insertions(+)
diff --git a/arch/x86/kvm/mmu/tdp_iter.h b/arch/x86/kvm/mmu/tdp_iter.h
index f0af385c56e0..9d982ccf4567 100644
--- a/arch/x86/kvm/mmu/tdp_iter.h
+++ b/arch/x86/kvm/mmu/tdp_iter.h
@@ -13,6 +13,12 @@
* to be zapped while holding mmu_lock for read, and to allow TLB flushes to be
* batched without having to collect the list of zapped SPs. Flows that can
* remove SPs must service pending TLB flushes prior to dropping RCU protection.
+ *
+ * The READ_ONCE() ensures that, if the SPTE points at a child shadow page, all
+ * fields in struct kvm_mmu_page will be read after the caller observes the
+ * present SPTE (KVM must check that the SPTE is present before following the
+ * SPTE's pfn to its associated shadow page). Pairs with the implicit memory
+ * barrier in tdp_mmu_set_spte_atomic().
*/
static inline u64 kvm_tdp_mmu_read_spte(tdp_ptep_t sptep)
{
diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
index 73eb28ed1f03..d1079fabe14c 100644
--- a/arch/x86/kvm/mmu/tdp_mmu.c
+++ b/arch/x86/kvm/mmu/tdp_mmu.c
@@ -658,6 +658,11 @@ static inline int tdp_mmu_set_spte_atomic(struct kvm *kvm,
lockdep_assert_held_read(&kvm->mmu_lock);
/*
+ * The atomic CMPXCHG64 provides an implicit memory barrier and ensures
+ * that, if the SPTE points at a shadow page, all struct kvm_mmu_page
+ * fields are visible to readers before the SPTE is marked present.
+ * Pairs with ordering guarantees provided by kvm_tdp_mmu_read_spte().
+ *
* Note, fast_pf_fix_direct_spte() can also modify TDP MMU SPTEs and
* does not hold the mmu_lock.
*/
--
2.37.2.672.g94769d06f0-goog
