Re: [PATCH v12 00/25] Drivers for Gunyah hypervisor

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 





On 4/24/2023 4:15 PM, Elliot Berman wrote:
Gunyah is a Type-1 hypervisor independent of any
high-level OS kernel, and runs in a higher CPU privilege level. It does
not depend on any lower-privileged OS kernel/code for its core
functionality. This increases its security and can support a much smaller
trusted computing base than a Type-2 hypervisor.

Gunyah is an open source hypervisor. The source repo is available at
https://github.com/quic/gunyah-hypervisor.

The diagram below shows the architecture.

::

          VM A                    VM B
      +-----+ +-----+  | +-----+ +-----+ +-----+
      |     | |     |  | |     | |     | |     |
  EL0 | APP | | APP |  | | APP | | APP | | APP |
      |     | |     |  | |     | |     | |     |
      +-----+ +-----+  | +-----+ +-----+ +-----+
  ---------------------|-------------------------
      +--------------+ | +----------------------+
      |              | | |                      |
  EL1 | Linux Kernel | | |Linux kernel/Other OS |   ...
      |              | | |                      |
      +--------------+ | +----------------------+
  --------hvc/smc------|------hvc/smc------------
      +----------------------------------------+
      |                                        |
  EL2 |            Gunyah Hypervisor           |
      |                                        |
      +----------------------------------------+

Gunyah provides these following features.

- Threads and Scheduling: The scheduler schedules virtual CPUs (VCPUs) on
physical CPUs and enables time-sharing of the CPUs.
- Memory Management: Gunyah tracks memory ownership and use of all memory
under its control. Memory partitioning between VMs is a fundamental
security feature.
- Interrupt Virtualization: All interrupts are handled in the hypervisor
and routed to the assigned VM.
- Inter-VM Communication: There are several different mechanisms provided
for communicating between VMs.
- Device Virtualization: Para-virtualization of devices is supported using
inter-VM communication. Low level system features and devices such as
interrupt controllers are supported with emulation where required.

This series adds the basic framework for detecting that Linux is running
under Gunyah as a virtual machine, communication with the Gunyah Resource
Manager, and a sample virtual machine manager capable of launching virtual machines.

The series relies on two other patches posted separately:
  - https://lore.kernel.org/all/20230213181832.3489174-1-quic_eberman@xxxxxxxxxxx/
  - https://lore.kernel.org/all/20230213232537.2040976-2-quic_eberman@xxxxxxxxxxx/

Changes in v12:
  - Stylistic/cosmetic tweaks suggested by Alex
  - Remove patch "virt: gunyah: Identify hypervisor version" and squash the
    check that we're running under a reasonable Gunyah hypervisor into RM driver
  - Refactor platform hooks into a separate module per suggestion from Srini
  - GFP_KERNEL_ACCOUNT and account_locked_vm() for page pinning
  - enum-ify related constant
The series is being applied onto the Android Common Kernel android14-6.1 branch and I've posted a series to bring the branch from v11 to v12. This would show what changed between the two versions:

https://android-review.googlesource.com/q/hashtag:%22gunyah-v12%22+(status:open%20OR%20status:merged)

  - Allow removal of VM functions by function-specific comparison -- specifically to allow
    removing irqfd by label only and not requiring original FD to be provided.


This last bullet point will be included in v13.

Changes in v11: https://lore.kernel.org/all/20230304010632.2127470-1-quic_eberman@xxxxxxxxxxx/
  - Rename struct gh_vm_dtb_config:gpa -> guest_phys_addr & overflow checks for this
  - More docstrings throughout
  - Make resp_buf and resp_buf_size optional
  - Replace deprecated idr with xarray
  - Refconting on misc device instead of RM's platform device
  - Renaming variables, structs, etc. from gunyah_ -> gh_
  - Drop removal of user mem regions
  - Drop mem_lend functionality; to converge with restricted_memfd later

Changes in v10: https://lore.kernel.org/all/20230214211229.3239350-1-quic_eberman@xxxxxxxxxxx/
  - Fix bisectability (end result of series is same, --fixups applied to wrong commits)
  - Convert GH_ERROR_* and GH_RM_ERROR_* to enums
  - Correct race condition between allocating/freeing user memory
  - Replace offsetof with struct_size
  - Series-wide renaming of functions to be more consistent
  - VM shutdown & restart support added in vCPU and VM Manager patches
  - Convert VM function name (string) to type (number)
  - Convert VM function argument to value (which could be a pointer) to remove memory wastage for arguments
  - Remove defensive checks of hypervisor correctness
  - Clean ups to ioeventfd as suggested by Srivatsa

Changes in v9: https://lore.kernel.org/all/20230120224627.4053418-1-quic_eberman@xxxxxxxxxxx/
  - Refactor Gunyah API flags to be exposed as feature flags at kernel level
  - Move mbox client cleanup into gunyah_msgq_remove()
  - Simplify gh_rm_call return value and response payload
  - Missing clean-up/error handling/little endian fixes as suggested by Srivatsa and Alex in v8 series

Changes in v8: https://lore.kernel.org/all/20221219225850.2397345-1-quic_eberman@xxxxxxxxxxx/
  - Treat VM manager as a library of RM
  - Add patches 21-28 as RFC to support proxy-scheduled vCPUs and necessary bits to support virtio
    from Gunyah userspace

Changes in v7: https://lore.kernel.org/all/20221121140009.2353512-1-quic_eberman@xxxxxxxxxxx/
  - Refactor to remove gunyah RM bus
  - Refactor allow multiple RM device instances
  - Bump UAPI to start at 0x0
  - Refactor QCOM SCM's platform hooks to allow CONFIG_QCOM_SCM=Y/CONFIG_GUNYAH=M combinations

Changes in v6: https://lore.kernel.org/all/20221026185846.3983888-1-quic_eberman@xxxxxxxxxxx/
  - *Replace gunyah-console with gunyah VM Manager*
  - Move include/asm-generic/gunyah.h into include/linux/gunyah.h
  - s/gunyah_msgq/gh_msgq/
  - Minor tweaks and documentation tidying based on comments from Jiri, Greg, Arnd, Dmitry, and Bagas.

Changes in v5: https://lore.kernel.org/all/20221011000840.289033-1-quic_eberman@xxxxxxxxxxx/
  - Dropped sysfs nodes
  - Switch from aux bus to Gunyah RM bus for the subdevices
  - Cleaning up RM console

Changes in v4: https://lore.kernel.org/all/20220928195633.2348848-1-quic_eberman@xxxxxxxxxxx/
  - Tidied up documentation throughout based on questions/feedback received
  - Switched message queue implementation to use mailboxes
  - Renamed "gunyah_device" as "gunyah_resource"

Changes in v3: https://lore.kernel.org/all/20220811214107.1074343-1-quic_eberman@xxxxxxxxxxx/
  - /Maintained/Supported/ in MAINTAINERS
  - Tidied up documentation throughout based on questions/feedback received
  - Moved hypercalls into arch/arm64/gunyah/; following hyper-v's implementation
  - Drop opaque typedefs
  - Move sysfs nodes under /sys/hypervisor/gunyah/
  - Moved Gunyah console driver to drivers/tty/
  - Reworked gh_device design to drop the Gunyah bus.

Changes in v2: https://lore.kernel.org/all/20220801211240.597859-1-quic_eberman@xxxxxxxxxxx/
  - DT bindings clean up
  - Switch hypercalls to follow SMCCC

v1: https://lore.kernel.org/all/20220223233729.1571114-1-quic_eberman@xxxxxxxxxxx/

Elliot Berman (25):
   docs: gunyah: Introduce Gunyah Hypervisor
   dt-bindings: Add binding for gunyah hypervisor
   gunyah: Common types and error codes for Gunyah hypercalls
   virt: gunyah: Add hypercalls to identify Gunyah
   virt: gunyah: msgq: Add hypercalls to send and receive messages
   mailbox: Add Gunyah message queue mailbox
   gunyah: rsc_mgr: Add resource manager RPC core
   gunyah: rsc_mgr: Add VM lifecycle RPC
   gunyah: vm_mgr: Introduce basic VM Manager
   gunyah: rsc_mgr: Add RPC for sharing memory
   gunyah: vm_mgr: Add/remove user memory regions
   gunyah: vm_mgr: Add ioctls to support basic non-proxy VM boot
   samples: Add sample userspace Gunyah VM Manager
   gunyah: rsc_mgr: Add platform ops on mem_lend/mem_reclaim
   firmware: qcom_scm: Register Gunyah platform ops
   docs: gunyah: Document Gunyah VM Manager
   virt: gunyah: Translate gh_rm_hyp_resource into gunyah_resource
   gunyah: vm_mgr: Add framework for VM Functions
   virt: gunyah: Add resource tickets
   virt: gunyah: Add IO handlers
   virt: gunyah: Add proxy-scheduled vCPUs
   virt: gunyah: Add hypercalls for sending doorbell
   virt: gunyah: Add irqfd interface
   virt: gunyah: Add ioeventfd
   MAINTAINERS: Add Gunyah hypervisor drivers section

  .../bindings/firmware/gunyah-hypervisor.yaml  |  82 ++
  .../userspace-api/ioctl/ioctl-number.rst      |   1 +
  Documentation/virt/gunyah/index.rst           | 114 +++
  Documentation/virt/gunyah/message-queue.rst   |  71 ++
  Documentation/virt/gunyah/vm-manager.rst      | 142 +++
  Documentation/virt/index.rst                  |   1 +
  MAINTAINERS                                   |  13 +
  arch/arm64/Kbuild                             |   1 +
  arch/arm64/gunyah/Makefile                    |   3 +
  arch/arm64/gunyah/gunyah_hypercall.c          | 140 +++
  arch/arm64/include/asm/gunyah.h               |  24 +
  drivers/mailbox/Makefile                      |   2 +
  drivers/mailbox/gunyah-msgq.c                 | 210 ++++
  drivers/virt/Kconfig                          |   2 +
  drivers/virt/Makefile                         |   1 +
  drivers/virt/gunyah/Kconfig                   |  59 ++
  drivers/virt/gunyah/Makefile                  |  11 +
  drivers/virt/gunyah/gunyah_ioeventfd.c        | 117 +++
  drivers/virt/gunyah/gunyah_irqfd.c            | 167 ++++
  drivers/virt/gunyah/gunyah_platform_hooks.c   |  80 ++
  drivers/virt/gunyah/gunyah_qcom.c             | 147 +++
  drivers/virt/gunyah/gunyah_vcpu.c             | 456 +++++++++
  drivers/virt/gunyah/rsc_mgr.c                 | 910 ++++++++++++++++++
  drivers/virt/gunyah/rsc_mgr.h                 |  19 +
  drivers/virt/gunyah/rsc_mgr_rpc.c             | 500 ++++++++++
  drivers/virt/gunyah/vm_mgr.c                  | 792 +++++++++++++++
  drivers/virt/gunyah/vm_mgr.h                  |  70 ++
  drivers/virt/gunyah/vm_mgr_mm.c               | 256 +++++
  include/linux/gunyah.h                        | 207 ++++
  include/linux/gunyah_rsc_mgr.h                | 162 ++++
  include/linux/gunyah_vm_mgr.h                 | 124 +++
  include/uapi/linux/gunyah.h                   | 293 ++++++
  samples/Kconfig                               |  10 +
  samples/Makefile                              |   1 +
  samples/gunyah/.gitignore                     |   2 +
  samples/gunyah/Makefile                       |   6 +
  samples/gunyah/gunyah_vmm.c                   | 270 ++++++
  samples/gunyah/sample_vm.dts                  |  68 ++
  38 files changed, 5534 insertions(+)
  create mode 100644 Documentation/devicetree/bindings/firmware/gunyah-hypervisor.yaml
  create mode 100644 Documentation/virt/gunyah/index.rst
  create mode 100644 Documentation/virt/gunyah/message-queue.rst
  create mode 100644 Documentation/virt/gunyah/vm-manager.rst
  create mode 100644 arch/arm64/gunyah/Makefile
  create mode 100644 arch/arm64/gunyah/gunyah_hypercall.c
  create mode 100644 arch/arm64/include/asm/gunyah.h
  create mode 100644 drivers/mailbox/gunyah-msgq.c
  create mode 100644 drivers/virt/gunyah/Kconfig
  create mode 100644 drivers/virt/gunyah/Makefile
  create mode 100644 drivers/virt/gunyah/gunyah_ioeventfd.c
  create mode 100644 drivers/virt/gunyah/gunyah_irqfd.c
  create mode 100644 drivers/virt/gunyah/gunyah_platform_hooks.c
  create mode 100644 drivers/virt/gunyah/gunyah_qcom.c
  create mode 100644 drivers/virt/gunyah/gunyah_vcpu.c
  create mode 100644 drivers/virt/gunyah/rsc_mgr.c
  create mode 100644 drivers/virt/gunyah/rsc_mgr.h
  create mode 100644 drivers/virt/gunyah/rsc_mgr_rpc.c
  create mode 100644 drivers/virt/gunyah/vm_mgr.c
  create mode 100644 drivers/virt/gunyah/vm_mgr.h
  create mode 100644 drivers/virt/gunyah/vm_mgr_mm.c
  create mode 100644 include/linux/gunyah.h
  create mode 100644 include/linux/gunyah_rsc_mgr.h
  create mode 100644 include/linux/gunyah_vm_mgr.h
  create mode 100644 include/uapi/linux/gunyah.h
  create mode 100644 samples/gunyah/.gitignore
  create mode 100644 samples/gunyah/Makefile
  create mode 100644 samples/gunyah/gunyah_vmm.c
  create mode 100644 samples/gunyah/sample_vm.dts


base-commit: d71ee0d6ee72138dd4fda646a955a15286c46544



[Index of Archives]     [Device Tree Compilter]     [Device Tree Spec]     [Linux Driver Backports]     [Video for Linux]     [Linux USB Devel]     [Linux PCI Devel]     [Linux Audio Users]     [Linux Kernel]     [Linux SCSI]     [XFree86]     [Yosemite Backpacking]


  Powered by Linux