Hello Marc, thanks again for your proposal. On Fri, 2023-03-24 at 14:46 +0000, Marc Zyngier wrote: > This series aims at satisfying multiple goals: > > - allow a VMM to atomically restore a timer offset for a whole VM > instead of updating the offset each time a vcpu get its counter > written > > - allow a VMM to save/restore the physical timer context, something > that we cannot do at the moment due to the lack of offsetting > > - provide a framework that is suitable for NV support, where we get > both global and per timer, per vcpu offsetting, and manage > interrupts in a less braindead way. > > We fix a couple of issues along the way, both from a stylistic and > correctness perspective. This results in a new per VM KVM API that > allows a global offset to be set at any point in time, overriding > both > of the timer counter writebacks. > > We also take this opportunity to rework the way IRQs are associated > with timers, something that was always a bit dodgy. This relies on a > new lock, which should disappear once Oliver's lock ordering series > is > merged (we can reuse the config_lock for this). > > This has been tested with nVHE, VHE and NV. I do not have access to > CNTPOFF-aware HW, but Colton managed to give it a go. Note that the > NV patches in this series are here to give a perspective on how this > gets used. > > I've updated the arch_timer selftest to allow an offset to be > provided > from the command line, and fixed a couple of glaring issues along the > way. > > Note that this is at best 6.4 material. I have a branch stashed at > [0] > and based on 6.3-rc3, as well as a minimal example of the use of the > API at [3] based on kvmtool. > > Simon: I'd appreciate some feedback as whether this change fits your > requirements, given that you brought this up the first place. The interface looks good to me. I have yet to deliver on my promise to test your patch series with our userspace; I am on leave this week, but I'll give your latest iteration a go next week. Thanks Simon > Thanks, > > M. > > * From v2 [2]: > > - Fixed 32bit handling of the physical counter when the offset is > non-zero > > - Dropped unused -O option from the selftest > > - Added lockdep_assert_held() to (un)lock_all_vcpus() > > - Reordered the last two patches > > - Added Colton's RBs, with thanks > > - Dropped the initial patch which has been merged > > - Rebased on 6.3-rc3 > > * From v1 [1]: > > - Switched from a dual offset to a single one which gets applied to > both virtual and physical counters. Which means that NV doesn't > behave oddly anymore by ignoring the virtual offset. > > - Some cosmetic repainting of the UAPI symbols > > - Added patches to rework the IRQ mapping to timers > > - Patch #1 on its way to Paolo > > - Rebased on 6.3-rc1 > > [0] > https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=kvm-arm64/timer-vm-offsets > [1] https://lore.kernel.org/r/20230216142123.2638675-1-maz@xxxxxxxxxx > [2] https://lore.kernel.org/r/20230313124837.2264882-5-maz@xxxxxxxxxx > [3] > https://git.kernel.org/pub/scm/linux/kernel/git/maz/kvmtool.git/commit/?h=zero-offset&id=3b1253073ee57c0d92baf7b214362829b487b8d5 > > Marc Zyngier (18): > KVM: arm64: timers: Use a per-vcpu, per-timer accumulator for > fractional ns > arm64: Add CNTPOFF_EL2 register definition > arm64: Add HAS_ECV_CNTPOFF capability > KVM: arm64: timers: Use CNTPOFF_EL2 to offset the physical timer > KVM: arm64: timers: Allow physical offset without CNTPOFF_EL2 > KVM: arm64: Expose {un,}lock_all_vcpus() to the rest of KVM > KVM: arm64: timers: Allow userspace to set the global counter > offset > KVM: arm64: timers: Allow save/restoring of the physical timer > KVM: arm64: timers: Rationalise per-vcpu timer init > KVM: arm64: timers: Abstract per-timer IRQ access > KVM: arm64: timers: Move the timer IRQs into arch_timer_vm_data > KVM: arm64: Abstract the number of valid timers per vcpu > KVM: arm64: Document KVM_ARM_SET_CNT_OFFSETS and co > KVM: arm64: nv: timers: Add a per-timer, per-vcpu offset > KVM: arm64: nv: timers: Support hyp timer emulation > KVM: arm64: selftests: Add physical timer registers to the sysreg > list > KVM: arm64: selftests: Deal with spurious timer interrupts > KVM: arm64: selftests: Augment existing timer test to handle > variable > offset > > Documentation/virt/kvm/api.rst | 38 ++ > arch/arm64/include/asm/kvm_host.h | 13 + > arch/arm64/include/asm/sysreg.h | 2 + > arch/arm64/include/uapi/asm/kvm.h | 11 + > arch/arm64/kernel/cpufeature.c | 11 + > arch/arm64/kvm/arch_timer.c | 539 ++++++++++++++-- > -- > arch/arm64/kvm/arm.c | 53 ++ > arch/arm64/kvm/guest.c | 29 +- > arch/arm64/kvm/hyp/nvhe/timer-sr.c | 18 +- > arch/arm64/kvm/hypercalls.c | 2 +- > arch/arm64/kvm/sys_regs.c | 9 + > arch/arm64/kvm/trace_arm.h | 6 +- > arch/arm64/kvm/vgic/vgic-kvm-device.c | 38 -- > arch/arm64/kvm/vgic/vgic.c | 15 + > arch/arm64/kvm/vgic/vgic.h | 3 - > arch/arm64/tools/cpucaps | 1 + > arch/arm64/tools/sysreg | 4 + > include/clocksource/arm_arch_timer.h | 1 + > include/kvm/arm_arch_timer.h | 36 +- > include/kvm/arm_vgic.h | 1 + > include/uapi/linux/kvm.h | 3 + > .../selftests/kvm/aarch64/arch_timer.c | 56 +- > .../selftests/kvm/aarch64/get-reg-list.c | 5 +- > 23 files changed, 690 insertions(+), 204 deletions(-) > > -- > 2.34.1 > Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
