On Tue, Jan 21, 2025, Nicolas Saenz Julienne wrote: > Hi Sean, > > On Fri Jan 17, 2025 at 9:48 PM UTC, Sean Christopherson wrote: > > On Wed, Oct 23, 2024, Paolo Bonzini wrote: > >> @@ -6398,6 +6415,46 @@ the capability to be present. > >> `flags` must currently be zero. > >> > >> > >> +.. _KVM_CREATE_PLANE: > >> + > >> +4.144 KVM_CREATE_PLANE > >> +---------------------- > >> + > >> +:Capability: KVM_CAP_PLANE > >> +:Architectures: none > >> +:Type: vm ioctl > >> +:Parameters: plane id > >> +:Returns: a VM fd that can be used to control the new plane. > >> + > >> +Creates a new *plane*, i.e. a separate privilege level for the > >> +virtual machine. Each plane has its own memory attributes, > >> +which can be used to enable more restricted permissions than > >> +what is allowed with ``KVM_SET_USER_MEMORY_REGION``. > >> + > >> +Each plane has a numeric id that is used when communicating > >> +with KVM through the :ref:`kvm_run <kvm_run>` struct. While > >> +KVM is currently agnostic to whether low ids are more or less > >> +privileged, it is expected that this will not always be the > >> +case in the future. For example KVM in the future may use > >> +the plane id when planes are supported by hardware (as is the > >> +case for VMPLs in AMD), or if KVM supports accelerated plane > >> +switch operations (as might be the case for Hyper-V VTLs). > >> + > >> +4.145 KVM_CREATE_VCPU_PLANE > >> +--------------------------- > >> + > >> +:Capability: KVM_CAP_PLANE > >> +:Architectures: none > >> +:Type: vm ioctl (non default plane) > >> +:Parameters: vcpu file descriptor for the default plane > >> +:Returns: a vCPU fd that can be used to control the new plane > >> + for the vCPU. > >> + > >> +Adds a vCPU to a plane; the new vCPU's id comes from the vCPU > >> +file descriptor that is passed in the argument. Note that > >> + because of how the API is defined, planes other than plane 0 > >> +can only have a subset of the ids that are available in plane 0. > > > > Hmm, was there a reason why we decided to add KVM_CREATE_VCPU_PLANE, as opposed > > to having KVM_CREATE_PLANE create vCPUs? IIRC, we talked about being able to > > provide the new FD, but that would be easy enough to handle in KVM_CREATE_PLANE, > > e.g. with an array of fds. > > IIRC we mentioned that there is nothing in the VSM spec preventing > higher VTLs from enabling a subset of vCPUs. That said, even the TLFS > mentions that doing so is not such a great idea (15.4 VTL Enablement): > > "Enable the target VTL on one or more virtual processors. [...] It is > recommended that all VPs have the same enabled VTLs. Having a VTL > enabled on some VPs (but not all) can lead to unexpected behavior." > > One thing I've been meaning to research is moving device emulation into > guest execution context by using VTLs. In that context, it might make > sense to only enable VTLs on specific vCPUs. But I'm only speculating. Creating vCPUs for a VTL in KVM doesn't need to _enable_ that VTL, and AIUI shouldn't enable the VTL, because HvCallEnablePartitionVtl "only" enables the VTL for the VM, HvCallEnableVpVtl is what fully enables the VTL for a given vCPU. What I am proposing is to create the KVM vCPU object(s) at KVM_CREATE_PLANE, purely to help avoid NULL pointer dereferences. Actually, since KVM will likely need uAPI to let userspace enable a VTL for a vCPU even if the vCPU object is auto-created, we could have KVM auto-create the objects transparently, i.e. still provide KVM_CREATE_VCPU_PLANE, but under the hood it would simply enable a flag and install the vCPU's file descriptor. > Otherwise, I cannot think of real world scenarios where this property is > needed. > > > k.g. is the expectation that userspace will create all planes before creating > > any vCPUs? > > The opposite really, VTLs can be initiated anytime during runtime. Oh, right. > > My concern with relying on userspace to create vCPUs is that it will mean KVM > > will need to support, or at least not blow up on, VMs with multiple planes, but > > only a subset of vCPUs at planes > 0. Given the snafus with vcpus_array, it's > > not at all hard to imagine scenarios where KVM tries to access a NULL vCPU in > > a different plane.
