On Thu, Aug 15, 2019 at 09:23:24AM -0600, Alex Williamson wrote: > Ok, fun day of trying to figure out which ranges are relevant, I've > narrowed it down to all of these: > > 0xffe00 > 0xfee00 > 0xfec00 APIC and I/O APIC stuff > 0xc1000 Assigned audio > 0x80a000 ? > 0x800000 GPU BAR > 0x100000 ? The APIC ranges are puzzling, I wouldn't expect their mappings to change. > ie. I can effective only say that sp->gfn values of 0x0, 0x40000, and > 0x80000 can take the continue branch without seeing bad behavior in the > VM. > > The assigned GPU has BARs at GPAs: > > 0xc0000000-0xc0ffffff > 0x800000000-0x808000000 > 0x808000000-0x809ffffff > > And the assigned companion audio function is at GPA: > > 0xc1080000-0xc1083fff > > Only one of those seems to align very well with a gfn base involved > here. The virtio ethernet has an mmio range at GPA 0x80a000000, > otherwise I don't find any other I/O devices coincident with the gfns > above. > > I'm running the VM with 2MB hugepages, but I believe the issue still > occurs with standard pages. When run with standard pages I see more > hits to gfn values 0, 0x40000, 0x80000, but the same number of hits to > the set above that cannot take the continue branch. I don't know if > that means anything. > > Any further ideas what to look for? Thanks, Maybe try isolating which memslot removal causes problems? E.g. flush the affected ranges if base_gfn == (xxx || yyy || zzz), otherwise flush only the memslot's gfns. Based on the log you sent a while back for gfn mismatches, I'm guessing the culprits are all GPU BARs, but it's probably worth confirming. That might also explain why gfn == 0x80000 can take the continue branch, i.e. if removing the corresponding memslot is what's causing problems, then it's being flushed and not actually taking the continue path. One other thought would be to force a call to kvm_flush_remote_tlbs(kvm), e.g. set flush=true just before the final kvm_mmu_remote_flush_or_zap(). Maybe it's a case where there are no SPTEs for the memslot, but the TLB flush is needed for some reason.
