In researching KVM_REG_ARM_TIMER_CNT, I discovered your commit 4b7a6bf
("target-arm: kvm: Differentiate registers based on write-back
levels"), which seems to limit when the KVM_REG_ARM_TIMER_CNT is used
to save time. Under what circumstances should this be saved in order
to provide a consistent view of wall clock time (as given by `date` in
the VM)? The commit refers to 'machine initialization or on vmload
operations', but I'm having difficulty figuring out what a vmload
operation is on ARM. Does this include resume-from-sleep/suspend?
On Fri, Nov 2, 2018 at 11:28 AM Miriam Zimmerman <mutexlox@xxxxxxxxxx> wrote:
>
> On Fri, Nov 2, 2018 at 7:34 AM Christoffer Dall
> <christoffer.dall@xxxxxxx> wrote:
> >
> > On Wed, Oct 31, 2018 at 11:49:05AM -0700, Miriam Zimmerman wrote:
> >
> > [...]
> >
> > >
> > > > CONFIG_KVM_GUEST fixes this by providing the guest with information on
> > > > the host time and informs it when the guest is paused (see
> > > > MSR_KVM_SYSTEM_TIME_NEW). Arm doesn't (yet) have para-virtualised time.
> > > >
> > > > On Arm, as far as I know, the guest's view of time is purely from the
> > > > virtual counter. Since nothing saves/restores this during the pause, the
> > > > counter continues to increment and the jump in time is visible to the guest.
> > > What is this virtual counter? How can one configure or enable it?
> >
> > The 'virtual counter' is an architectural concept in the Arm
> > architecture, not a virtual (in the software VM sense) counter.
> >
> > The Arm Generic Timer Architecture defines two counters:
> > - The physical counter, accessed via CNTPCT_EL0
> > - The virtual counter, accessed via CNTVCT_EL0
> >
> > And a number of timers. All timers are associated with the physical
> > counter, except the 'EL1 Virtual Timer' which uses the count in the
> > virtual counter for its comparison.
> >
> > The virtual counter yields the value of the physical counter minus an
> > offset.
> >
> > That offset is controlled by a hypervisor running at EL2 which can
> > program the offset value in CNTVOFF_EL2.
> >
> > The basic idea is that a VM can use the virtual timer/counter to keep
> > track of some notion of virtual time, and the physical timer/counter to
> > keep track of something that always relates to wall-clock time.
> >
> > This all breaks quite horribly with migration where the physical counter
> > is not meaningful across physical systems, and where the counter
> > frequencies can vary.
> >
> > But we can use the offset for the virtual counter to communicate some
> > other measure of time to VMs.
>
> That makes sense. Thanks for the primer! I haven't done any ARM kernel
> work before, so this is all great to know.
>
> > > My understanding of what's going on now is that the guest kernel right
> > > now has roughly no awareness that it's inside of a VM, so it just
> > > tracks time using the assumption that it is native and is unaware that
> > > the host ever suspends. (It just doesn't see any CPU time then.)
> > >
> > > >
> > > > Whether the guest sees time progress depends on what happens to that
> > > > counter during suspend. kvmtool pause/resume simply prevents the vCPU
> > > > threads from continuing, but the system counter is still running.
> > > >
> > > > If you save the state of a VM to a file then the counter value is
> > > > saved/restored so the guest won't see any change in time.
> > > I don't believe that we're explicitly saving any state to a file on
> > > suspend right now, though I could be wrong; I haven't looked into that
> > > codepath yet.
> >
> > The key is whether the userspace program that controls the KVM VM
> > (kvmtool, QEMU, crosvm) uses the KVM_REG_ARM_TIMER_CNT ioctl to save the
> > VM view of virtual time, and to retore that at a later time.
> >
> > KVM adjusts the CNTVOFF_EL2 for the VM on which the ioctl is executed to
> > represent the value written by userspace to the VM when the VM reads
> > CNTVCT_EL0.
> >
>
> Ah hah. I haven't looked yet, but I bet this is the missing piece.
> Thanks very much, Christoffer!
>
> >
> > Thanks,
> >
> > Christoffer
_______________________________________________
kvmarm mailing list
kvmarm@xxxxxxxxxxxxxxxxxxxxx
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
