Hi Oliver, Adding Andrew and Peter to the discussion. On Tue, 24 Aug 2021 22:15:03 +0100, Oliver Upton <oupton@xxxxxxxxxx> wrote: > > Hey folks, > > Ricardo and I were discussing hypercall support in KVM/arm64 and > something seems to be a bit problematic. I do not see anywhere that KVM > requires opt-in from the VMM to expose new hypercalls to the guest. To > name a few, the TRNG and KVM PTP hypercalls are unconditionally provided > to the guest. > > Exposing new hypercalls to guests in this manner seems very unsafe to > me. Suppose an operator is trying to upgrade from kernel N to kernel > N+1, which brings in the new 'widget' hypercall. Guests are live > migrated onto the N+1 kernel, but the operator finds a defect that > warrants a kernel rollback. VMs are then migrated from kernel N+1 -> N. > Any guests that discovered the 'widget' hypercall are likely going to > get fussy _very_ quickly on the old kernel. This goes against what we decided to support for the *only* publicly available VMM that cares about save/restore, which is that we only move forward and don't rollback. Hypercalls are the least of your worries, and there is a whole range of other architectural features that will have also appeared/disappeared (your own CNTPOFF series is a glaring example of this). I appreciate that you may have different considerations, but I felt that it was important to state *why* this is the way it is. > > Really, we need to ensure that the exposed guest ABI is > backwards-compatible. Running a VMM that is blissfully unaware of the > 'widget' hypercall should not implicitly expose it to its guest on a new > kernel. > > This conversation was in the context of devising a new UAPI that allows > VMMs to trap hypercalls to userspace. While such an interface would > easily work around the issue, the onus of ABI compatibility lies with > the kernel. > > So, this is all a long-winded way of asking: how do we dig ourselves out > of this situation, and how to we avoid it happening again in the future? > I believe the answer to both is to have new VM capabilities for sets of > hypercalls exposed to the guest. Unfortunately, the unconditional > exposure of TRNG and PTP hypercalls is ABI now, so we'd have to provide > an opt-out at this point. For anything new, require opt-in from the VMM > before we give it to the guest. > > Have I missed something blatantly obvious, or do others see this as an > issue as well? I'll reply with an example of adding opt-out for PTP. > I'm sure other hypercalls could be handled similarly. Why do we need this? For future hypercalls, we could have some buy-in capabilities. For existing ones, it is too late, and negative features are just too horrible. For KVM-specific hypercalls, we could get the VMM to save/restore the bitmap of supported functions. That would be "less horrible". This could be implemented using extra "firmware pseudo-registers" such as the ones described in Documentation/virt/kvm/arm/psci.rst. Thanks, M. -- Without deviation from the norm, progress is not possible.
