2016-10-17 07:47-0200, Marcelo Tosatti: > On Fri, Oct 14, 2016 at 06:20:31PM -0300, Eduardo Habkost wrote: >> I have been wondering: should we allow live migration with the >> invtsc flag enabled, if TSC scaling is available on the >> destination? > > TSC scaling and invtsc flag, yes. Yes, if we have well synchronized time between hosts, then we might be able to migrate with a TSC shift that cannot be perceived by the guest. Unless the VM also has a migratable assigned PCI device that uses ART, because we have no protocol to update the setting of ART (in CPUID), so we should keep migration forbidden then. >> For reference, this is what the Intel SDM says about invtsc: >> >> The time stamp counter in newer processors may support an >> enhancement, referred to as invariant TSC. Processor’s support >> for invariant TSC is indicated by CPUID.80000007H:EDX[8]. >> >> The invariant TSC will run at a constant rate in all ACPI P-, >> C-. and T-states. This is the architectural behavior moving >> forward. On processors with invariant TSC support, the OS may >> use the TSC for wall clock timer services (instead of ACPI or >> HPET timers). TSC reads are much more efficient and do not >> incur the overhead associated with a ring transition or access >> to a platform resource. > > Yes. The blockage happened for different reasons: > > 1) Migration: to host with different TSC frequency. We shouldn't have done this even now when emulating anything newer than Pentium 4, because those CPUs have constant TSC, which only lacks the guarantee that it doesn't stop in deep C-states: For [a list of processors we emulate]: the time-stamp counter increments at a constant rate. That rate may be set by the maximum core-clock to bus-clock ratio of the processor or may be set by the maximum resolved frequency at which the processor is booted. The maximum resolved frequency may differ from the processor base frequency, see Section 18.18.2 for more detail. On certain processors, the TSC frequency may not be the same as the frequency in the brand string. The specific processor configuration determines the behavior. Constant TSC behavior ensures that the duration of each clock tick is uniform and supports the use of the TSC as a wall clock timer even if the processor core changes frequency. This is the architectural behavior moving forward. Invariant TSC is more useful, though, so more applications would break when migrating to a different TSC frequency. > 2) Savevm: It is not safe to use the TSC for wall clock timer > services. With constant TSC, we could argue that a shift to deep C-state happened and paused TSC, which is not a good behavior, but somewhat defensible. > By allowing savevm, you make a commitment to allow a feature > at the expense of not complying with the spec (specifically the " > the OS may use the TSC for wall clock timer services", because the > TSC stops relative to realtime for the duration of the savevm stop > window). Yep, we should at least guesstimate the TSC to allow the guest to resume with as small TSC-shift as possible and check that hosts were somewhat synchronized with UTC (or something we choose for time). > But since Linux guests use kvmclock and Windows guests use Hyper-V > enlightenment, it should be fine to disable 2). > > There is a bug open for this, btw: > https://bugzilla.redhat.com/show_bug.cgi?id=1353073 These people should be happy with just live-migrations, so can't we just keep savevm forbidden? -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
