On Thu, Aug 30, 2012 at 5:22 AM, Xiao Guangrong <xiaoguangrong@xxxxxxxxxxxxxxxxxx> wrote: > On 08/28/2012 11:30 AM, Felix wrote: >> Xiao Guangrong <xiaoguangrong <at> linux.vnet.ibm.com> writes: >> >>> >>> On 07/31/2012 01:18 AM, Sunil wrote: >>>> Hello List, >>>> >>>> I am a KVM newbie and studying KVM mmu code. >>>> >>>> On the existing guest, I am trying to track all guest writes by >>>> marking page table entry as read-only in EPT entry [ I am using Intel >>>> machine with vmx and ept support ]. Looks like EPT support re-uses >>>> shadow page table(SPT) code and hence some of SPT routines. >>>> >>>> I was thinking of below possible approach. Use pte_list_walk() to >>>> traverse through list of sptes and use mmu_spte_update() to flip the >>>> PT_WRITABLE_MASK flag. But all SPTEs are not part of any single list; >>>> but on separate lists (based on gfn, page level, memory_slot). So, >>>> recording all the faulted guest GFN and then using above method work ? >>>> >>> >>> There are two ways to write-protect all sptes: >>> - use kvm_mmu_slot_remove_write_access() on all memslots >>> - walk the shadow page cache to get the shadow pages in the highest level >>> (level = 4 on EPT), then write-protect its entries. >>> >>> If you just want to do it for the specified gfn, you can use >>> rmap_write_protect(). >>> >>> Just inquisitive, what is your purpose? :) >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe kvm" in >>> the body of a message to majordomo <at> vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >>> >> Hi, Guangrong, >> >> I have done similar things like Sunil did. Simply for study purpose. However, I >> found some very weird situations. Basically, in the guest vm, I allocate a chunk >> of memory (with size of a page) in a user level program. Through a guest kernel >> level module and my self defined hypercall, I pass the gva of this memory to >> kvm. Then I try different methods in the hypercall handler to write protect this >> page of memory. You can see that I want to write protect it through ETP instead >> of write protected in the guest page tables. >> >> 1. I use kvm_mmu_gva_to_gpa_read to translate the gva into gpa. Based on the >> function, kvm_mmu_get_spte_hierarchy(vcpu, gpa, spte[4]), I change the codes to >> read sptep (the pointer to spte) instead of spte, so I can modify the spte >> corresponding to this gpa. What I observe is that if I modify spte[0] (I think >> this is the lowest level page table entry corresponding to EPT table; I can >> successfully modify it as the changes are reflected in the result of calling >> kvm_mmu_get_spte_hierarchy again), but my user level program in vm can still >> write to this page. >> >> In your this blog post, you mentioned (the shadow pages in the highest level >> (level = 4 on EPT)), I don't understand this part. Does this mean I have to >> modify spte[3] instead of spte[0]? I just try modify spte[1] and spte[3], both >> can cause vmexit. So I am totally confused about the meaning of level used in >> shadow page table and its relations to shadow page table. Can you help me to >> understand this? >> >> 2. As suggested by this post, I also use rmap_write_protect() to write protect >> this page. With kvm_mmu_get_spte_hierarchy(vcpu, gpa, spte[4]), I still can see >> that spte[0] gives me xxxxxx005 such result, this means that the function is >> called successfully. But still I can write to this page. >> >> I even try the function kvm_age_hva() to remove this spte, this gives me 0 of >> spte[0], but I still can write to this page. So I am further confused about the >> level used in the shadow page? >> > > kvm_mmu_get_spte_hierarchy get sptes out of mmu-lock, you can hold spin_lock(&vcpu->kvm->mmu_lock) > and use for_each_shadow_entry instead. And, after change, did you flush all tlbs? I do apply the lock in my codes and I do flush tlb. > > If it can not work, please post your code. > Here is my codes. The modifications are made in x86/x86.c in KVM_HC_HL_EPTPER is my hypercall number. Method 1: int kvm_emulate_hypercall(struct kvm_vcpu *vcpu){ ................ case KVM_HC_HL_EPTPER : //// This method is not working localGpa = kvm_mmu_gva_to_gpa_read(vcpu, a0, &localEx); if(localGpa == UNMAPPED_GVA){ printk("read is not correct\n"); return -KVM_ENOSYS; } hl_kvm_mmu_update_spte(vcpu, localGpa, 5); hl_result = kvm_mmu_get_spte_hierarchy(vcpu, localGpa, hl_sptes); printk("after changes return result is %d , gpa: %llx sptes: %llx , %llx , %llx , %llx \n", hl_result, localGpa, hl_sptes[0], hl_sptes[1], hl_sptes[2], hl_sptes[3]); kvm_flush_remote_tlbs(vcpu->kvm); ................... } The function hl_kvm_mmu_update_spte is defined as int hl_kvm_mmu_update_spte(struct kvm_vcpu *vcpu, u64 addr, u64 mask) { struct kvm_shadow_walk_iterator iterator; int nr_sptes = 0; u64 sptes[4]; u64* sptep[4]; u64 localMask = 0xFFFFFFFFFFFFFFF8; /// 1000 spin_lock(&vcpu->kvm->mmu_lock); for_each_shadow_entry(vcpu, addr, iterator) { sptes[iterator.level-1] = *iterator.sptep; sptep[iterator.level-1] = iterator.sptep; nr_sptes++; if (!is_shadow_present_pte(*iterator.sptep)) break; } sptes[0] = sptes[0] & localMask; sptes[0] = sptes[0] | mask ; __set_spte(sptep[0], sptes[0]); //update_spte(sptep[0], sptes[0]); /* sptes[1] = sptes[1] & localMask; sptes[1] = sptes[1] | mask ; update_spte(sptep[1], sptes[1]); */ /* sptes[3] = sptes[3] & localMask; sptes[3] = sptes[3] | mask ; update_spte(sptep[3], sptes[3]); */ spin_unlock(&vcpu->kvm->mmu_lock); return nr_sptes; } The execution results are from kern.log xxxx kernel: [ 4371.002579] hypercall f002, a71000 xxxx kernel: [ 4371.002581] after changes return result is 4 , gpa: 723ae000 sptes: 16c7bd275 , 1304c7007 , 136d6f007 , 13cc88007 I find that if I write to this page, actually the write protected permission bit is set as writable again. I am not quite sure why. Method 2: int kvm_emulate_hypercall(struct kvm_vcpu *vcpu){ ................ case KVM_HC_HL_EPTPER : //// This method is not working localGpa = kvm_mmu_gva_to_gpa_read(vcpu, a0, &localEx); localGfn = gpa_to_gfn(localGpa); spin_lock(&vcpu->kvm->mmu_lock); hl_result = rmap_write_protect(vcpu->kvm, localGfn); printk("local gfn is %llx , result of kvm_age_hva is %d\n", localGfn, hl_result); kvm_flush_remote_tlbs(vcpu->kvm); spin_unlock(&vcpu->kvm->mmu_lock); hl_result = kvm_mmu_get_spte_hierarchy(vcpu, localGpa, hl_sptes); printk("return result is %d , gpa: %llx sptes: %llx , %llx , %llx , %llx \n", hl_result, localGpa, hl_sptes[0], hl_sptes[1], hl_sptes[2], hl_sptes[3]); ................... } The execution results are: xxxx kernel: [ 4044.020816] hypercall f002, 1201000 xxxx kernel: [ 4044.020819] local gfn is 70280 , result of kvm_age_hva is 1 xxxx kernel: [ 4044.020823] return result is 4 , gpa: 70280000 sptes: 13c2aa275 , 1304ff007 , 15eb3d007 , 15eb3e007 My feeling is seems that I have to modify something else instead of spte alone. Thanks for your help, Felix -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
