On Sun, May 21, 2023 at 11:44:36AM +0800, Robert Hoo wrote: > On 5/9/2023 9:53 PM, Yan Zhao wrote: > > Keep a per-VM MTRR state and point it to the MTRR state of vCPU 0. > > > > This is a preparation patch for KVM to reference a per-VM guest MTRR > > to decide memory type of EPT leaf entries when noncoherent DMA is present. > > > > Though each vCPU has its own MTRR state, MTRR states should be > > consistent across each VM, which is demanded as in Intel's SDM > > "In a multiprocessor system using a processor in the P6 family or a more > > recent family, each processor MUST use the identical MTRR memory map so > > that software will have a consistent view of memory." > > > > Therefore, when memory type of EPT leaf entry needs to honor guest MTRR, > > a per-VM version of guest MTRR can be referenced. > > > > Each vCPU still has its own MTRR state field to keep guest rdmsr() > > returning the right value when there's lag of MTRR update for each vCPU. > > > Can we get rid of per-vCPU MTRR state copies and just have this per-VM state > only? therefore can simplify implementation and avoid hazard of > inconsistency among per-VPU MTRR states. > > I see in SDM, it notes: > "In multiple processor systems, the operating system must maintain MTRR > consistency between all the processors in the system (that is, all > processors must use the same MTRR values). The P6 and more recent processor > families provide no hardware support for maintaining this consistency." > > leaving each vCPU's MTRR is just to fully mimic HW? > Yes, leaving each vCPU's MTRR to mimic HW. As also suggested in SDM, the guest OS manipulates MTRRs in this way: for each online CPUs { disable MTRR update fixed/var MTRR ranges enable MTRR } Guest OS needs to access memory only after this full pattern. So, I think there should not have "hazard of inconsistency among per-VPU MTRR states". I want to have per-VM MTRR state is because I want to reduce unnessary EPT zap, which costs quite a lot cpu cycles even when the EPT is empty. In this patch, per-VM MTRR pointer is used to point to vCPU 0's MTRR state, so that it can save some memory to keep the MTRR state. But I found out that it would only work when vCPU 0 (boot processor) is always online (which is not true for x86 under some configration). I'll try to find out lowest online vCPU and keep a per-VM copy of MTRR state in next version. Thanks!