Hi everyone, Here are some patches to enable SEV-TIO (aka TDISP, aka secure VFIO) on AMD Turin. The basic idea is to allow DMA to/from encrypted memory of SNP VMs and secure MMIO in SNP VMs (i.e. with Cbit set) as well. These include both guest and host support. QEMU also requires some patches, links below. The patches are organized as: 01..06 - preparing the host OS; 07 - new TSM module; 08 - add PSP SEV TIO ABI (IDE should start working at this point); 09..14 - add KVM support (TDI binding, MMIO faulting, etc); 15..19 - guest changes (the rest of SEV TIO ABI, DMA, secure MMIO). 20, 21 - some helpers for guest OS to use encrypted MMIO This is based on a merge of ee3248f9f8d6 Lukas Wunner spdm: Allow control of next requester nonce through sysfs 85ef1ac03941 (AMDESE/snp-host-latest) 4 days ago Michael Roth [TEMP] KVM: guest_memfd: Update gmem_prep are hook to handle partially-allocated folios Please comment. Thanks. Thanks, SEV TIO tree prototype ====================== Goal ---- Support secure PCI devices pass through to confidential VMs. The support is defined by PCIe 6, SPDM, TDISP (not AMD) and SEV TIO specification (by AMD). SEV TIO spec: https://www.amd.com/content/dam/amd/en/documents/epyc-technical-docs/specifications/58271_0_70.pdf Whitepaper: https://www.amd.com/content/dam/amd/en/documents/epyc-business-docs/white-papers/sev-tio-whitepaper.pdf GHCB spec update is coming. The changeset adds: - a generic TSM.ko module; makes necessary changes to: - ccp (interface to AMD secure platform processor, "PSP", works on the host), - sev-guest (interace to PSP for confidential SNP guest), - kvm_amd, - memfd, - vfio kvm device, - kvm-x86. Acronyms -------- TEE - Trusted Execution Environments, a concept of managing trust between the host and devices TSM - TEE Security Manager (TSM), an entity which ensures security on the host PSP - AMD platform secure processor (also "ASP", "AMD-SP"), acts as TSM on AMD. SEV TIO - the TIO protocol implemented by the PSP and used by the host GHCB - guest/host communication block - a protocol for guest-to-host communication via a shared page CMA, IDE, TDISP, TVM, DSM, SPDM, DOE Code ---- Written with AMD SEV SNP in mind, TSM is the PSP and therefore no much of IDE/TDISP is left for the host or guest OS. Add a common module to expose various data objects in the same way in host and guest OS. Provide a know on the host to enable IDE encryption. Add another version of Guest Request for secure guest<->PSP communication. Enable secure DMA by: - configuring vTOM in a secure DTE via the PSP to cover the entire guest RAM; - mapping all private memory pages in IOMMU just like as they were shared (requires hacking iommufd); - skipping various enforcements of non-SME or SWIOTLB in the guest; No mixed share+private DMA supported within the same IOMMU. Enable secure MMIO by: - configuring RMP entries via the PSP; - adding necessary helpers for mapping MMIO with the Cbit set; - hacking the KVM #PF handler to allow private MMIO failts. Based on the latest upstream KVM (at the moment it is kvm-coco-queue). Workflow -------- 1. Boot host OS. 2. "Connect" the physical device. 3. Bind a VF to VFIO-PCI. 4. Run QEMU _without_ the device yet. 5. Hotplug the VF to the VM. 6. (if not already) Load the device driver. 7. Right after the BusMaster is enabled, tsm.ko performs secure DMA and MMIO setup. 8. Run tests, for example: sudo ./pcimem/pcimem /sys/bus/pci/devices/0000\:01\:00.0/resource4_enc 0 w*4 0xabcd Assumptions ----------- This requires hotpligging into the VM vs passing the device via the command line as VFIO maps all guest memory as the device init step which is too soon as SNP LAUNCH UPDATE happens later and will fail if VFIO maps private memory before that. This requires the BME hack as MMIO and BusMaster enable bits cannot be 0 after MMIO validation is done and there are moments in the guest OS booting process when this appens. SVSM could help addressing these (not implemented at the moment). QEMU advertises TEE-IO capability to the VM. An additional x-tio flag is added to vfio-pci. TODOs ----- Deal with PCI reset. Hot unplug+plug? Power states too. Do better generalization, the current code heavily uses SEV TIO defined structures in supposedly generic code. Fix the documentation comments of SEV TIO structures. Git trees --------- https://github.com/AMDESE/linux-kvm/tree/tio https://github.com/AMDESE/qemu/tree/tio Alexey Kardashevskiy (21): tsm-report: Rename module to reflect what it does pci/doe: Define protocol types and make those public pci: Define TEE-IO bit in PCIe device capabilities PCI/IDE: Define Integrity and Data Encryption (IDE) extended capability crypto/ccp: Make some SEV helpers public crypto: ccp: Enable SEV-TIO feature in the PSP when supported pci/tdisp: Introduce tsm module crypto/ccp: Implement SEV TIO firmware interface kvm: Export kvm_vm_set_mem_attributes vfio: Export helper to get vfio_device from fd KVM: SEV: Add TIO VMGEXIT and bind TDI KVM: IOMMUFD: MEMFD: Map private pages KVM: X86: Handle private MMIO as shared RFC: iommu/iommufd/amd: Add IOMMU_HWPT_TRUSTED flag, tweak DTE's DomainID, IOTLB coco/sev-guest: Allow multiple source files in the driver coco/sev-guest: Make SEV-to-PSP request helpers public coco/sev-guest: Implement the guest side of things RFC: pci: Add BUS_NOTIFY_PCI_BUS_MASTER event sev-guest: Stop changing encrypted page state for TDISP devices pci: Allow encrypted MMIO mapping via sysfs pci: Define pci_iomap_range_encrypted drivers/crypto/ccp/Makefile | 2 + drivers/pci/Makefile | 1 + drivers/virt/coco/Makefile | 3 +- drivers/virt/coco/sev-guest/Makefile | 1 + arch/x86/include/asm/kvm-x86-ops.h | 2 + arch/x86/include/asm/kvm_host.h | 2 + arch/x86/include/asm/sev.h | 23 + arch/x86/include/uapi/asm/svm.h | 2 + arch/x86/kvm/svm/svm.h | 2 + drivers/crypto/ccp/sev-dev-tio.h | 105 ++ drivers/crypto/ccp/sev-dev.h | 4 + drivers/iommu/amd/amd_iommu_types.h | 2 + drivers/iommu/iommufd/io_pagetable.h | 3 + drivers/iommu/iommufd/iommufd_private.h | 4 + drivers/virt/coco/sev-guest/sev-guest.h | 56 + include/asm-generic/pci_iomap.h | 4 + include/linux/device.h | 5 + include/linux/device/bus.h | 3 + include/linux/dma-direct.h | 4 + include/linux/iommufd.h | 6 + include/linux/kvm_host.h | 70 + include/linux/pci-doe.h | 4 + include/linux/pci-ide.h | 18 + include/linux/pci.h | 2 +- include/linux/psp-sev.h | 116 +- include/linux/swiotlb.h | 4 + include/linux/tsm-report.h | 113 ++ include/linux/tsm.h | 337 +++-- include/linux/vfio.h | 1 + include/uapi/linux/iommufd.h | 1 + include/uapi/linux/kvm.h | 29 + include/uapi/linux/pci_regs.h | 77 +- include/uapi/linux/psp-sev.h | 4 +- arch/x86/coco/sev/core.c | 11 + arch/x86/kvm/mmu/mmu.c | 6 +- arch/x86/kvm/svm/sev.c | 217 +++ arch/x86/kvm/svm/svm.c | 3 + arch/x86/kvm/x86.c | 12 + arch/x86/mm/mem_encrypt.c | 5 + arch/x86/virt/svm/sev.c | 23 +- drivers/crypto/ccp/sev-dev-tio.c | 1565 ++++++++++++++++++++ drivers/crypto/ccp/sev-dev-tsm.c | 397 +++++ drivers/crypto/ccp/sev-dev.c | 87 +- drivers/iommu/amd/iommu.c | 20 +- drivers/iommu/iommufd/hw_pagetable.c | 4 + drivers/iommu/iommufd/io_pagetable.c | 2 + drivers/iommu/iommufd/main.c | 21 + drivers/iommu/iommufd/pages.c | 94 +- drivers/pci/doe.c | 2 - drivers/pci/ide.c | 186 +++ drivers/pci/iomap.c | 24 + drivers/pci/mmap.c | 11 +- drivers/pci/pci-sysfs.c | 27 +- drivers/pci/pci.c | 3 + drivers/pci/proc.c | 2 +- drivers/vfio/vfio_main.c | 13 + drivers/virt/coco/sev-guest/{sev-guest.c => sev_guest.c} | 68 +- drivers/virt/coco/sev-guest/sev_guest_tio.c | 513 +++++++ drivers/virt/coco/tdx-guest/tdx-guest.c | 8 +- drivers/virt/coco/tsm-report.c | 512 +++++++ drivers/virt/coco/tsm.c | 1542 ++++++++++++++----- virt/kvm/guest_memfd.c | 40 + virt/kvm/kvm_main.c | 4 +- virt/kvm/vfio.c | 197 ++- Documentation/virt/coco/tsm.rst | 62 + MAINTAINERS | 4 +- arch/x86/kvm/Kconfig | 1 + drivers/pci/Kconfig | 4 + drivers/virt/coco/Kconfig | 11 + 69 files changed, 6163 insertions(+), 548 deletions(-) create mode 100644 drivers/crypto/ccp/sev-dev-tio.h create mode 100644 drivers/virt/coco/sev-guest/sev-guest.h create mode 100644 include/linux/pci-ide.h create mode 100644 include/linux/tsm-report.h create mode 100644 drivers/crypto/ccp/sev-dev-tio.c create mode 100644 drivers/crypto/ccp/sev-dev-tsm.c create mode 100644 drivers/pci/ide.c rename drivers/virt/coco/sev-guest/{sev-guest.c => sev_guest.c} (96%) create mode 100644 drivers/virt/coco/sev-guest/sev_guest_tio.c create mode 100644 drivers/virt/coco/tsm-report.c create mode 100644 Documentation/virt/coco/tsm.rst -- 2.45.2
