The existing TDP MMU methods to handle dirty logging are vcpu-agnostic
since they can be driven by MMU notifiers and other non-vcpu-specific
events in addition to page faults. However this means that the TDP MMU
is not benefiting from the new vcpu->lru_slot_index. Fix that by special
casing dirty logging in tdp_mmu_map_handle_target_level.
This improves "Populate memory time" in dirty_log_perf_test by 5%:
Command | Before | After
------------------------------- | ---------------- | -------------
./dirty_log_perf_test -v64 -x64 | 5.472321072s | 5.169832886s
Signed-off-by: David Matlack <dmatlack@xxxxxxxxxx>
---
arch/x86/kvm/mmu/tdp_mmu.c | 15 ++++++++++++---
1 file changed, 12 insertions(+), 3 deletions(-)
diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
index 43f12f5d12c0..1467f99c846d 100644
--- a/arch/x86/kvm/mmu/tdp_mmu.c
+++ b/arch/x86/kvm/mmu/tdp_mmu.c
@@ -929,10 +929,19 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu, int write,
map_writable, !shadow_accessed_mask,
&new_spte);
- if (new_spte == iter->old_spte)
+ if (new_spte == iter->old_spte) {
ret = RET_PF_SPURIOUS;
- else if (!tdp_mmu_set_spte_atomic(vcpu->kvm, iter, new_spte))
- return RET_PF_RETRY;
+ } else {
+ if (!tdp_mmu_set_spte_atomic_no_dirty_log(vcpu->kvm, iter, new_spte))
+ return RET_PF_RETRY;
+
+ /*
+ * Mark the gfn dirty here rather that through the vcpu-agnostic
+ * handle_changed_spte_dirty_log to leverage vcpu->lru_slot_index.
+ */
+ if (is_writable_pte(new_spte))
+ kvm_vcpu_mark_page_dirty(vcpu, iter->gfn);
+ }
/*
* If the page fault was caused by a write but the page is write
--
2.32.0.554.ge1b32706d8-goog
