Hi Marc, On Tue, Feb 8, 2022 at 1:46 AM Marc Zyngier <maz@xxxxxxxxxx> wrote: > > > KVM currently restricts the vcpu features to be unified across vcpus, > > > but that's only an implementation choice. > > > > But that implementation choice has become ABI, no? How could support > > for asymmetry be added without requiring userspace opt-in or breaking > > existing VMMs that depend on feature unification? > > Of course, you'd need some sort of advertising of this new behaviour. > > One thing I would like to add to the current state of thing is an > indication of whether the effects of a sysreg being written from > userspace are global or local to a vcpu. You'd need a new capability, > and an extra flag added to the encoding of each register. Ah. I think that is a much more reasonable fit then. VMMs unaware of this can continue to migrate new bits (albeit at the cost of potentially higher lock contention for the per-VM stuff), and those that do can reap the benefits of writing such attributes exactly once. [...] > > > A device means yet another configuration and migration API. Don't you > > > think we have enough of those? The complexity of KVM/arm64 userspace > > > API is already insane, and extremely fragile. Adding to it will be a > > > validation nightmare (it already is, and I don't see anyone actively > > > helping with it). > > > > It seems equally fragile to introduce VM-wide serialization to vCPU > > UAPI that we know is in the live migration critical path for _any_ > > VMM. Without requiring userspace changes for all the new widgets under > > discussion we're effectively forcing VMMs to do something suboptimal. > > I'm perfectly happy with suboptimal to start with, and find ways to > make it better once we have a clear path forward. I just don't want to > conflate the two. Fair. My initial concern was that it didn't seem as though there was much room for growth/improvement with the one reg UAPI, but your suggestion definitely provides a ramp out to handle VM state once per VM. Thanks for following up :) -- Oliver
