Re: [RFC] Support for Arm CCA VMs on Linux

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Wed, Jan 10, 2024 at 11:41:09AM +0000, Suzuki K Poulose wrote:
> Hi Itaru,
> 
> On 10/01/2024 05:40, Itaru Kitayama wrote:
> > On Mon, Oct 02, 2023 at 01:43:11PM +0100, Suzuki K Poulose wrote:
> > > Hi,
> > > 
> > > 
> > > > We are happy to announce the early RFC version of the Arm
> > > > Confidential Compute Architecture (CCA) support for the Linux
> > > > stack. The intention is to seek early feedback in the following areas:
> > > >   * KVM integration of the Arm CCA
> > > >   * KVM UABI for managing the Realms, seeking to generalise the operations
> > > >     wherever possible with other Confidential Compute solutions.
> > > >     Note: This version doesn't support Guest Private memory, which will be added
> > > >     later (see below).
> > > >   * Linux Guest support for Realms
> > > > 
> > > 
> > > We have updated the stack for Arm CCA Linux support to RMM-v1.0-EAC2 (See links)
> > > We are not posting the patches for review yet, as we plan to update our
> > > stack to support the latest RMM-v1.0 specification, which includes some
> > > functional changes to support PSCI monitoring by the VMM along with other
> > > minor changes. All relevant components are updated on a new branch "rmm-v1.0-eac2"
> > > Guest-mem support is not included, but is in progress.
> > > 
> > > Change log :
> > >   - KVM RMI support updated to v1.0-eac2, with optimisations to stage2 tear down
> > >   - Guest (Linux and kvm-unit-test) support for RSI compliant to v1.0-eac2
> > >   - SVE, PMU support for Realms
> > > 
> > > kvmtool :
> > >    - Dropped no-compat and switched to --loglevel (merged upstream)
> > >    - Support for SVE, --sve-vl for vector length
> > > 
> > > > Arm CCA Introduction
> > > > =====================
> > > > 
> > > > The Arm CCA is a reference software architecture and implementation that builds
> > > > on the Realm Management Extension (RME), enabling the execution of Virtual
> > > > machines, while preventing access by more privileged software, such as hypervisor.
> > > > The Arm CCA allows the hypervisor to control the VM, but removes the right for
> > > > access to the code, register state or data that is used by VM.
> > > > More information on the architecture is available here[0].
> > > > 
> > > >      Arm CCA Reference Software Architecture
> > > > 
> > > >          Realm World    ||    Normal World   ||  Secure World  ||
> > > >                         ||        |          ||                ||
> > > >   EL0 x-------x         || x----x | x------x ||                ||
> > > >       | Realm |         || |    | | |      | ||                ||
> > > >       |       |         || | VM | | |      | ||                ||
> > > >   ----|  VM*  |---------||-|    |---|      |-||----------------||
> > > >       |       |         || |    | | |  H   | ||                ||
> > > >   EL1 x-------x         || x----x | |      | ||                ||
> > > >           ^             ||        | |  o   | ||                ||
> > > >           |             ||        | |      | ||                ||
> > > >   ------- R*------------------------|  s  -|---------------------
> > > >           S             ||          |      | ||                ||
> > > >           I             ||          |  t   | ||                ||
> > > >           |             ||          |      | ||                ||
> > > >           v             ||          x------x ||                ||
> > > >   EL2    RMM*           ||              ^    ||                ||
> > > >           ^             ||              |    ||                ||
> > > >   ========|=============================|========================
> > > >           |                             | SMC
> > > >           x--------- *RMI* -------------x
> > > > 
> > > >   EL3                   Root World
> > > >                         EL3 Firmware
> > > >   ===============================================================
> > > > Where :
> > > >   RMM - Realm Management Monitor
> > > >   RMI - Realm Management Interface
> > > >   RSI - Realm Service Interface
> > > >   SMC - Secure Monitor Call
> > > > 
> > > > RME introduces a new security state "Realm world", in addition to the
> > > > traditional Secure and Non-Secure states. The Arm CCA defines a new component,
> > > > Realm Management Monitor (RMM) that runs at R-EL2. This is a standard piece of
> > > > firmware, verified, installed and loaded by the EL3 firmware (e.g, TF-A), at
> > > > system boot.
> > > > 
> > > > The RMM provides standard interfaces - Realm Management Interface (RMI) - to the
> > > > Normal world hypervisor to manage the VMs running in the Realm world (also called
> > > > Realms in short). These are exposed via SMC and are routed through the EL3
> > > > firmwre.
> > > > The RMI interface includes:
> > > >    - Move a physical page from the Normal world to the Realm world
> > > >    - Creating a Realm with requested parameters, tracked via Realm Descriptor (RD)
> > > >    - Creating VCPUs aka Realm Execution Context (REC), with initial register state.
> > > >    - Create stage2 translation table at any level.
> > > >    - Load initial images into Realm Memory from normal world memory
> > > >    - Schedule RECs (vCPUs) and handle exits
> > > >    - Inject virtual interrupts into the Realm
> > > >    - Service stage2 runtime faults with pages (provided by host, scrubbed by RMM).
> > > >    - Create "shared" mappings that can be accessed by VMM/Hyp.
> > > >    - Reclaim the memory allocated for the RAM and RTTs (Realm Translation Tables)
> > > > 
> > > > However v1.0 of RMM specifications doesn't support:
> > > >   - Paging protected memory of a Realm VM. Thus the pages backing the protected
> > > >     memory region must be pinned.
> > > >   - Live migration of Realms.
> > > >   - Trusted Device assignment.
> > > >   - Physical interrupt backed Virtual interrupts for Realms
> > > > 
> > > > RMM also provides certain services to the Realms via SMC, called Realm Service
> > > > Interface (RSI). These include:
> > > >   - Realm Guest Configuration.
> > > >   - Attestation & Measurement services
> > > >   - Managing the state of an Intermediate Physical Address (IPA aka GPA) page.
> > > >   - Host Call service (Communication with the Normal world Hypervisor)
> > > > 
> > > > The specifications for the RMM software is currently at *v1.0-Beta2* and the
> > > > latest version is available here [1].
> > > > 
> > > > The Trusted Firmware foundation has an implementation of the RMM - TF-RMM -
> > > > available here [3].
> > > > 
> > > > Implementation
> > > > =================
> > > > 
> > > > This version of the stack is based on the RMM specification v1.0-Beta0[2], with
> > > > following exceptions :
> > > >    - TF-RMM/KVM currently doesn't support the optional features of PMU,
> > > >       SVE and Self-hosted debug (coming soon).
> > > >    - The RSI_HOST_CALL structure alignment requirement is reduced to match
> > > >       RMM v1.0 Beta1
> > > >    - RMI/RSI version numbers do not match the RMM spec. This will be
> > > >      resolved once the spec/implementation is complete, across TF-RMM+Linux stack.
> > > > 
> > > > We plan to update the stack to support the latest version of the RMMv1.0 spec
> > > > in the coming revisions.
> > > > 
> > > > This release includes the following components :
> > > > 
> > > >   a) Linux Kernel
> > > >       i) Host / KVM support - Support for driving the Realms via RMI. This is
> > > >       dependent on running in the Kernel at EL2 (aka VHE mode). Also provides
> > > >       UABI for VMMs to manage the Realm VMs. The support is restricted to 4K page
> > > >       size, matching the Stage2 granule supported by RMM. The VMM is responsible
> > > >       for making sure the guest memory is locked.
> > > > 
> > > >         TODO: Guest Private memory[10] integration - We have been following the
> > > >         series and support will be added once it is merged upstream.
> > > >       ii) Guest support - Support for a Linux Kernel to run in the Realm VM at
> > > >       Realm-EL1, using RSI services. This includes virtio support (virtio-v1.0
> > > >       only). All I/O are treated as non-secure/shared.
> > > >   c) kvmtool - VMM changes required to manage Realm VMs. No guest private memory
> > > >      as mentioned above.
> > > >   d) kvm-unit-tests - Support for running in Realms along with additional tests
> > > >      for RSI ABI.
> > > > 
> > > > Running the stack
> > > > ====================
> > > > 
> > > > To run/test the stack, you would need the following components :
> > > > 
> > > > 1) FVP Base AEM RevC model with FEAT_RME support [4]
> > > > 2) TF-A firmware for EL3 [5]
> > > > 3) TF-A RMM for R-EL2 [3]
> > > > 4) Linux Kernel [6]
> > > > 5) kvmtool [7]
> > > > 6) kvm-unit-tests [8]
> > > > 
> > > > Instructions for building the firmware components and running the model are
> > > > available here [9]. Once, the host kernel is booted, a Realm can be launched by
> > > > invoking the `lkvm` commad as follows:
> > > > 
> > > >   $ lkvm run --realm 				 \
> > > > 	 --measurement-algo=["sha256", "sha512"] \
> > > > 	 --disable-sve				 \
> > > 
> > > As noted above, this is no longer required.
> > > 
> > > > 	 <normal-vm-options>
> > > > 
> > > > Where:
> > > >   * --measurement-algo (Optional) specifies the algorithm selected for creating the
> > > >     initial measurements by the RMM for this Realm (defaults to sha256).
> > > >   * GICv3 is mandatory for the Realms.
> > > >   * SVE is not yet supported in the TF-RMM, and thus must be disabled using
> > > >     --disable-sve
> > > > 
> > > > You may also run the kvm-unit-tests inside the Realm world, using the similar
> > > > options as above.
> > > > 
> > > > 
> > > > Links
> > > > ============
> > > > 
> > > > [0] Arm CCA Landing page (See Key Resources section for various documentations)
> > > >      https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture
> > > > 
> > > > [1] RMM Specification Latest
> > > >      https://developer.arm.com/documentation/den0137/latest
> > > > 
> > > > [2] RMM v1.0-Beta0 specification
> > > >      https://developer.arm.com/documentation/den0137/1-0bet0/
> > > 
> > >   EAC2 spec: https://developer.arm.com/documentation/den0137/1-0eac2/
> > > > 
> > > > [3] Trusted Firmware RMM - TF-RMM
> > > >      https://www.trustedfirmware.org/projects/tf-rmm/
> > > >      GIT: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
> > > > 
> > > > [4] FVP Base RevC AEM Model (available on x86_64 / Arm64 Linux)
> > > >      https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platforms
> > > > 
> > > > [5] Trusted Firmware for A class
> > > >      https://www.trustedfirmware.org/projects/tf-a/ >>>
> > > > [6] Linux kernel support for Arm-CCA
> > > >      https://gitlab.arm.com/linux-arm/linux-cca
> > > >      Host Support branch:	cca-host/rfc-v1
> > > 
> > > Update branch : cca-host/rmm-v1.0-eac2
> > > 
> > > >      Guest Support branch:	cca-guest/rfc-v1
> > > 
> > > Update branch : cca-guest/rmm-v1.0-eac2
> > > 
> > > Combined tree for host and guest is also available at: "cca-full/rmm-v1.0-eac2"
> > > 
> > > > 
> > > > [7] kvmtool support for Arm CCA
> > > >      https://gitlab.arm.com/linux-arm/kvmtool-cca cca/rfc-v1
> > > 
> > > Update branch : cca/rmm-v1.0-eac2
> > > 
> > > > 
> > > > [8] kvm-unit-tests support for Arm CCA
> > > >      https://gitlab.arm.com/linux-arm/kvm-unit-tests-cca  cca/rfc-v1
> > > > 
> > > 
> > > Update branch : cca/rmm-v1.0-eac2
> > > 
> > > 
> > > Suzuki
> > > 
> > > > [9] Instructions for Building Firmware components and running the model, see
> > > >      section 4.19.2 "Building and running TF-A with RME"
> > > >      https://trustedfirmware-a.readthedocs.io/en/latest/components/realm-management-extension.html#building-and-running-tf-a-with-rme
> > > > 
> > > > [10] fd based Guest Private memory for KVM
> > > >     https://lkml.kernel.org/r/20221202061347.1070246-1-chao.p.peng@xxxxxxxxxxxxxxx
> > > 
> > > 
> > > 
> > > 
> > > 
> > > Cc: Alexandru Elisei <alexandru.elisei@xxxxxxx>
> > > Cc: Andrew Jones <andrew.jones@xxxxxxxxx>
> > > Cc: Catalin Marinas <catalin.marinas@xxxxxxx>
> > > Cc: Chao Peng <chao.p.peng@xxxxxxxxxxxxxxx>
> > > Cc: Christoffer Dall <christoffer.dall@xxxxxxx>
> > > Cc: Fuad Tabba <tabba@xxxxxxxxxx>
> > > Cc: Jonathan Cameron <jonathan.cameron@xxxxxxxxxx>
> > > Cc: James Morse <james.morse@xxxxxxx>
> > > Cc: Jean-Philippe Brucker <jean-philippe@xxxxxxxxxx>
> > > Cc: Joey Gouly <Joey.Gouly@xxxxxxx>
> > > Cc: Marc Zyngier <maz@xxxxxxxxxx>
> > > Cc: Mark Rutland <mark.rutland@xxxxxxx>
> > > Cc: Oliver Upton <oliver.upton@xxxxxxxxx>
> > > Cc: Paolo Bonzini <pbonzini@xxxxxxxxxx>
> > > Cc: Quentin Perret <qperret@xxxxxxxxxx>
> > > Cc: Sean Christopherson <seanjc@xxxxxxxxxx>
> > > Cc: Steven Price <steven.price@xxxxxxx>
> > > Cc: Thomas Huth <thuth@xxxxxxxxxx>
> > > Cc: Will Deacon <will@xxxxxxxxxx>
> > > Cc: Zenghui Yu <yuzenghui@xxxxxxxxxx>
> > > To: linux-coco@xxxxxxxxxxxxxxx
> > > To: kvmarm@xxxxxxxxxxxxxxx
> > > Cc: linux-arm-kernel@xxxxxxxxxxxxxxxxxxx
> > > To: linux-kernel@xxxxxxxxxxxxxxx
> > > To: kvm@xxxxxxxxxxxxxxx
> > 
> > Suzuki,
> > Any update to the Arm CCA series (v3?) since last October?
> 
> Yes, we now have a version that supports the final RMM-v1.0
> specification (RMM-v1.0-EAC5). We also have the UEFI EDK2 firmware
> support for Guests in Realm world.
> 
> We are planning to post the changes for review in the v6.8-rc cycle. We
> are trying to integrate the guest_mem support (available in v6.8-rc1) as
> well as reusing some of the arm64 kvm generic interface for configuring
> the Realm parameters (e.g., PMU, SVE_VL etc).
> 
> Here is a version that is missing the items mentioned above, based
> on v6.7-rc4, if anyone would like to try.
> 
> Also, the easiest way to get the components built and model kick started
> is using the shrinkwrap [6] tool, using the cca-3world configuration.
> The tool pulls all the required software components, builds (including
> the buildroot for rootfs) and can run a model using these built
> components.

Hi Suzuki,

This is great news! I've just booted you guys WIP Linux kernel through
shrinkwrap (cca-3world.yaml) without an issue. 
Many thanks to Ryan who delivered an extremely handy tool to us. 

Thanks,
Itaru.

> 
> 
> 
> [0] Linux Repo:
>       Where: git@xxxxxxxxxxxxxxxxxx:linux-arm/linux-cca.git
>       KVM Support branch: cca-host/rmm-v1.0-eac5
>       Linux Guest branch: cca-guest/rmm-v1.0-eac5
>       Full stack branch:  cca-full/rmm-v1.0-eac5
> 
> [1] kvmtool Repo:
>       Where: git@xxxxxxxxxxxxxxxxxx:linux-arm/kvmtool-cca.git
>       Branch: cca/rmm-v1.0-eac5
> 
> [2] kvm-unit-tests Repo:
>       Where: git@xxxxxxxxxxxxxxxxxx:linux-arm/kvm-unit-tests-cca.git
>       Branch: cca/rmm-v1.0-eac5
> 
> [3] UEFI Guest firmware:
>       edk2:     https://git.gitlab.arm.com/linux-arm/edk2-cca.git
>       revision: 2802_arm_cca_rmm-v1.0-eac5
> 
>       edk2-platforms:
> https://git.gitlab.arm.com/linux-arm/edk2-platforms-cca.git
>       revision:       2802_arm_cca_rmm-v1.0-eac5
> 
> 
> [4] RMM Repo:
>       Where: https://git.trustedfirmware.org/TF-RMM/tf-rmm.git
>       tag : tf-rmm-v0.4.0
> 
> [5] TF-A repo:
>       Where: https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git
>       Tag: v2.10
> 
> 
> [6] https://shrinkwrap.docs.arm.com/en/latest/
>     config: cca-3world.yaml
> 
> Kind regards
> Suzuki
> 
> 




[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux