On Fri, May 26, 2023 at 12:39:03PM +0100, Daniel P. Berrangé wrote: > The 'virt-what' program prints facts that reflect the hypervisor that > the guest is running under. > > The new complementary 'virt-what-cvm' program prints facts that reflect > the confidential virtualization technology the guest is running under, > if any. > > It is kept as a separate tool, rather than incorporating the facts into > 'virt-what' output because it is considering a different aspect of the > virtualization. Furthermore there are specific security concerns around > the usage of facts reported by 'virt-what-cvm'. > > The tool has been tested in a number of environments > > * Azure confidential guest with AMD SEV-SNP (GA) > * Azure confidential guest with Intel TDX (technology preview) > * Fedora 37 QEMU/KVM guest with AMD SEV (GA) > * Fedora 37 QEMU/KVM guest with AMD SEV-ES (GA) > * Fedora 38 QEMU/KVM guest with AMD SEV-SNP + SVSM (devel snapshot) > > Signed-off-by: Daniel P. Berrangé <berrange@xxxxxxxxxx> Upstream in: 8056047..52c833c I added a second commit which cross-references the new manual page from the old one. I wonder if there's any way to add tests for this? The existing tool is tested using a kind of "virtual root" scheme, but that only works because of the specifics of the shell script and the fact that we can fake the output of virt-what-cpuid-helper in the virtual root (hence don't have to fake actual CPUID). Rich. > .gitignore | 3 + > Makefile.am | 12 +- > configure.ac | 3 + > virt-what-cvm.c | 404 ++++++++++++++++++++++++++++++++++++++++++++++ > virt-what-cvm.pod | 195 ++++++++++++++++++++++ > 5 files changed, 613 insertions(+), 4 deletions(-) > create mode 100644 virt-what-cvm.c > create mode 100644 virt-what-cvm.pod > > diff --git a/.gitignore b/.gitignore > index 4833fd6..ba897a1 100644 > --- a/.gitignore > +++ b/.gitignore > @@ -26,5 +26,8 @@ Makefile.in > /test-driver > /virt-what > /virt-what-cpuid-helper > +/virt-what-cvm > +/virt-what-cvm.1 > +/virt-what-cvm.txt > /virt-what.1 > /virt-what.txt > diff --git a/Makefile.am b/Makefile.am > index 5435132..2050bef 100644 > --- a/Makefile.am > +++ b/Makefile.am > @@ -24,20 +24,24 @@ EXTRA_DIST = .gitignore virt-what.in virt-what.pod > SUBDIRS = . tests > > sbin_SCRIPTS = virt-what > +sbin_PROGRAMS = virt-what-cvm > libexec_PROGRAMS = virt-what-cpuid-helper > if HOST_CPU_IA64 > libexec_PROGRAMS += virt-what-ia64-xen-rdtsc-test > endif > > +virt_what_cvm_LDADD = $(TPM2_TSS_LIBS) > +virt_what_cvm_CFLAGS = $(TPM2_TSS_CFLAGS) > + > if HAVE_POD2MAN > > -CLEANFILES += virt-what.1 virt-what.txt > -man_MANS = virt-what.1 > +CLEANFILES += virt-what.1 virt-what-cvm.1 virt-what.txt virt-what-cvm.txt > +man_MANS = virt-what.1 virt-what-cvm.1 > > -virt-what.1: virt-what.pod > +%.1: %.pod > pod2man -c "Virtualization Support" --release "$(PACKAGE)-$(VERSION)" \ > $? > $@ > -virt-what.txt: virt-what.pod > +%.txt: %.pod > pod2text $? > $@ > > endif > diff --git a/configure.ac b/configure.ac > index a28a716..77b7665 100644 > --- a/configure.ac > +++ b/configure.ac > @@ -32,6 +32,9 @@ dnl Architecture we are compiling for. > AC_CANONICAL_HOST > AM_CONDITIONAL([HOST_CPU_IA64], [ test "x$host_cpu" = "xia64" ]) > > +PKG_HAVE_DEFINE_WITH_MODULES(TPM2_TSS, tss2-esys, [tpm2-tss package]) > + > + > dnl List of tests. > tests="\ > alibaba-cloud-arm \ > diff --git a/virt-what-cvm.c b/virt-what-cvm.c > new file mode 100644 > index 0000000..407efb4 > --- /dev/null > +++ b/virt-what-cvm.c > @@ -0,0 +1,404 @@ > +/* virt-what-cvm-helper: Are we running inside confidential VM > + * Copyright (C) 2023 Red Hat Inc. > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation; either version 2 of the License, or > + * (at your option) any later version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program; if not, write to the Free Software > + * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. > + */ > + > +#include "config.h" > + > +#include <stdio.h> > +#include <stdlib.h> > +#include <stdint.h> > +#include <string.h> > +#include <stdbool.h> > +#include <fcntl.h> > +#include <unistd.h> > +#include <getopt.h> > +#ifdef HAVE_TPM2_TSS > +#include <tss2/tss2_esys.h> > +#include <assert.h> > +#endif > + > +static bool dodebug = false; > + > +#define debug(...) do { if (dodebug) fprintf(stderr, __VA_ARGS__); } while(0) > + > +/* > + * AMD64 Architecture Programmer’s Manual Volume 3: > + * General-Purpose and System Instructions. > + * Chapter: E4.1 - Maximum Extended Function Number and Vendor String > + * https://www.amd.com/system/files/TechDocs/24594.pdf > + */ > +#define CPUID_GET_HIGHEST_FUNCTION 0x80000000 > + > +/* > + * AMD64 Architecture Programmer’s Manual Volume 3: > + * General-Purpose and System Instructions. > + * Chapter: E4.17 - Encrypted Memory Capabilities > + * https://www.amd.com/system/files/TechDocs/24594.pdf > + */ > +#define CPUID_AMD_GET_ENCRYPTED_MEMORY_CAPABILITIES 0x8000001f > + > +/* > + * AMD64 Architecture Programmer’s Manual Volume 3: > + * General-Purpose and System Instructions. > + * Chapter: 15.34.10 - SEV_STATUS MSR > + * https://www.amd.com/system/files/TechDocs/24593.pdf > + */ > +#define MSR_AMD64_SEV 0xc0010131 > + > +/* > + * Intel® TDX Module v1.5 Base Architecture Specification > + * Chapter: 11.2 > + * https://www.intel.com/content/www/us/en/content-details/733575/intel-tdx-module-v1-5-base-architecture-specification.html > + */ > + > +#define CPUID_INTEL_TDX_ENUMERATION 0x21 > + > + > +#define CPUID_SIG_AMD "AuthenticAMD" > +#define CPUID_SIG_INTEL "GenuineIntel" > +#define CPUID_SIG_INTEL_TDX "IntelTDX " > + > +/* > + * This TPM NV data format is not explicitly documented anywhere, > + * but the header definition is present in code at: > + * > + * https://github.com/kinvolk/azure-cvm-tooling/blob/main/az-snp-vtpm/src/hcl.rs > + */ > +#define TPM_AZURE_HCLA_REPORT_INDEX 0x01400001 > + > +struct TPMAzureHCLAHeader { > + uint32_t signature; > + uint32_t version; > + uint32_t report_len; > + uint32_t report_type; > + uint32_t unknown[4]; > +}; > + > +/* The bytes for "HCLA" */ > +#define TPM_AZURE_HCLA_SIGNATURE 0x414C4348 > +#define TPM_AZURE_HCLA_VERSION 0x1 > +#define TPM_AZURE_HCLA_REPORT_TYPE_SNP 0x2 > + > +#if defined(__x86_64__) > + > +#ifdef HAVE_TPM2_TSS > +static char * > +tpm_nvread(uint32_t nvindex, size_t *retlen) > +{ > + TSS2_RC rc; > + ESYS_CONTEXT *ctx = NULL; > + ESYS_TR primary = ESYS_TR_NONE; > + ESYS_TR session = ESYS_TR_NONE; > + ESYS_TR nvobj = ESYS_TR_NONE; > + TPM2B_NV_PUBLIC *pubData = NULL; > + TPMT_SYM_DEF sym = { > + .algorithm = TPM2_ALG_AES, > + .keyBits = { .aes = 128 }, > + .mode = { .aes = TPM2_ALG_CFB } > + }; > + char *ret; > + size_t retwant; > + > + rc = Esys_Initialize(&ctx, NULL, NULL); > + if (rc != TSS2_RC_SUCCESS) > + return NULL; > + > + rc = Esys_Startup(ctx, TPM2_SU_CLEAR); > + debug("tpm startup %d\n", rc); > + if (rc != TSS2_RC_SUCCESS) > + goto error; > + > + rc = Esys_StartAuthSession(ctx, ESYS_TR_NONE, ESYS_TR_NONE, > + ESYS_TR_NONE, ESYS_TR_NONE, ESYS_TR_NONE, > + NULL, 0, > + &sym, TPM2_ALG_SHA256, &session); > + debug("tpm auth session %d\n", rc); > + if (rc != TSS2_RC_SUCCESS) > + goto error; > + > + rc = Esys_TR_FromTPMPublic(ctx, nvindex, ESYS_TR_NONE, > + ESYS_TR_NONE, ESYS_TR_NONE, &nvobj); > + debug("tpm from public %d\n", rc); > + if (rc != TSS2_RC_SUCCESS) > + goto error; > + > + rc = Esys_NV_ReadPublic(ctx, nvobj, ESYS_TR_NONE, > + ESYS_TR_NONE, ESYS_TR_NONE, > + &pubData, NULL); > + debug("tpm read public %d\n", rc); > + if (rc != TPM2_RC_SUCCESS) > + goto error; > + > + retwant = pubData->nvPublic.dataSize; > + free(pubData); > + *retlen = 0; > + ret = malloc(retwant); > + assert(ret); > + while (*retlen < retwant) { > + size_t want = retwant - *retlen; > + TPM2B_MAX_NV_BUFFER *data = NULL; > + if (want > 1024) > + want = 1024; > + rc = Esys_NV_Read(ctx, ESYS_TR_RH_OWNER, nvobj, session, ESYS_TR_NONE, ESYS_TR_NONE, > + want, *retlen, &data); > + debug("tpm nv read %d\n", rc); > + if (rc != TPM2_RC_SUCCESS) { > + free(ret); > + goto error; > + } > + > + memcpy(ret + *retlen, data->buffer, data->size); > + *retlen += data->size; > + free(data); > + } > + > + return ret; > + > + error: > + if (nvobj != ESYS_TR_NONE) > + Esys_FlushContext(ctx, nvobj); > + if (session != ESYS_TR_NONE) > + Esys_FlushContext(ctx, session); > + if (primary != ESYS_TR_NONE) > + Esys_FlushContext(ctx, primary); > + Esys_Finalize(&ctx); > + *retlen = 0; > + return NULL; > +} > +#else /* ! HAVE_TPM2_TSS */ > +static char * > +tpm_nvread(uint32_t nvindex, size_t *retlen) > +{ > + return NULL; > +} > +#endif /* ! HAVE_TPM2_TSS */ > + > +/* Copied from the Linux kernel definition in > + * arch/x86/include/asm/processor.h > + */ > +static inline void > +cpuid (uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx) > +{ > + debug("CPUID func %x %x\n", *eax, *ecx); > + asm volatile ("cpuid" > + : "=a" (*eax), "=b" (*ebx), "=c" (*ecx), "=d" (*edx) > + : "0" (*eax), "2" (*ecx) > + : "memory"); > + debug("CPUID result %x %x %x %x\n", *eax, *ebx, *ecx, *edx); > +} > + > + > +static uint32_t > +cpuid_leaf (uint32_t eax, char *sig) > +{ > + uint32_t *sig32 = (uint32_t *) sig; > + > + cpuid (&eax, &sig32[0], &sig32[2], &sig32[1]); > + sig[12] = 0; /* \0-terminate the string to make string comparison possible */ > + debug("CPUID sig %s\n", sig); > + return eax; > +} > + > +#define MSR_DEVICE "/dev/cpu/0/msr" > + > +static uint64_t > +msr (off_t index) > +{ > + uint64_t ret; > + int fd = open (MSR_DEVICE, O_RDONLY); > + if (fd < 0) { > + debug ("Cannot open MSR device %s", MSR_DEVICE); > + return 0; > + } > + > + if (pread (fd, &ret, sizeof(ret), index) != sizeof(ret)) > + ret = 0; > + > + close (fd); > + > + debug ("MSR %llx result %llx\n", (unsigned long long)index, > + (unsigned long long)ret); > + return ret; > +} > + > +bool > +cpu_sig_amd_azure (void) > +{ > + size_t datalen = 0; > + char *data = tpm_nvread(TPM_AZURE_HCLA_REPORT_INDEX, &datalen); > + struct TPMAzureHCLAHeader *header = (struct TPMAzureHCLAHeader *)data; > + bool ret; > + > + if (!data) > + return false; > + > + if (datalen < sizeof(struct TPMAzureHCLAHeader)) { > + debug ("TPM data len is too small to be an Azure HCLA report"); > + return false; > + } > + > + debug ("Azure TPM HCLA report header sig %x ver %x type %x\n", > + header->signature, header->version, header->report_type); > + > + ret = (header->signature == TPM_AZURE_HCLA_SIGNATURE && > + header->version == TPM_AZURE_HCLA_VERSION && > + header->report_type == TPM_AZURE_HCLA_REPORT_TYPE_SNP); > + debug ("Azure TPM HCLA report present ? %d\n", ret); > + > + free(data); > + return ret; > +} > + > +static void > +cpu_sig_amd (void) > +{ > + uint32_t eax, ebx, ecx, edx; > + uint64_t msrval; > + > + eax = CPUID_GET_HIGHEST_FUNCTION; > + ebx = ecx = edx = 0; > + > + cpuid (&eax, &ebx, &ecx, &edx); > + > + if (eax < CPUID_AMD_GET_ENCRYPTED_MEMORY_CAPABILITIES) > + return; > + > + eax = CPUID_AMD_GET_ENCRYPTED_MEMORY_CAPABILITIES; > + ebx = ecx = edx = 0; > + > + cpuid (&eax, &ebx, &ecx, &edx); > + > + /* bit 1 == CPU supports SEV feature > + * > + * Note, Azure blocks this CPUID leaf from its SEV-SNP > + * guests, so we must fallback to probing the TPM which > + * exposes a SEV-SNP attestation report as evidence. > + */ > + if (!(eax & (1 << 1))) { > + debug ("No sev in CPUID, try azure TPM NV\n"); > + > + if (cpu_sig_amd_azure()) { > + puts ("amd-sev-snp"); > + puts ("azure-hcl"); > + } else { > + debug("No azure TPM NV\n"); > + } > + return; > + } > + > + msrval = msr (MSR_AMD64_SEV); > + > + /* Test reverse order, since the SEV-SNP bit implies > + * the SEV-ES bit, which implies the SEV bit */ > + if (msrval & (1 << 2)) { > + puts ("amd-sev-snp"); > + } else if (msrval & (1 << 1)) { > + puts ("amd-sev-es"); > + } else if (msrval & (1 << 0)) { > + puts ("amd-sev"); > + } > +} > + > +static void > +cpu_sig_intel (void) > +{ > + uint32_t eax, ebx, ecx, edx; > + char sig[13]; > + > + eax = CPUID_GET_HIGHEST_FUNCTION; > + ebx = ecx = edx = 0; > + > + cpuid (&eax, &ebx, &ecx, &edx); > + debug ("CPUID max function: %x %x %x %x\n", eax, ebx, ecx,edx); > + > + if (eax < CPUID_INTEL_TDX_ENUMERATION) > + return; > + > + memset (sig, 0, sizeof sig); > + cpuid_leaf (CPUID_INTEL_TDX_ENUMERATION, sig); > + > + if (memcmp (sig, CPUID_SIG_INTEL_TDX, sizeof(sig)) == 0) > + puts ("intel-tdx"); > +} > + > +static void > +cpu_sig (void) > +{ > + char sig[13]; > + > + memset (sig, 0, sizeof sig); > + cpuid_leaf (0, sig); > + > + if (memcmp (sig, CPUID_SIG_AMD, sizeof(sig)) == 0) > + cpu_sig_amd (); > + else if (memcmp (sig, CPUID_SIG_INTEL, sizeof(sig)) == 0) > + cpu_sig_intel (); > +} > + > +#else /* !x86_64 */ > + > +static void > +cpu_sig (void) > +{ > + /* nothing for other architectures */ > +} > + > +#endif > + > +int > +main(int argc, char **argv) > +{ > + int c; > + > + while (true) { > + int option_index = 0; > + static struct option long_options[] = { > + {"debug", no_argument, 0, 'd' }, > + {"version", no_argument, 0, 'v' }, > + {"help", no_argument, 0, 'h'}, > + {0, 0, 0, 0 } > + }; > + > + c = getopt_long(argc, argv, "dvh", > + long_options, &option_index); > + if (c == -1) > + break; > + > + switch (c) { > + case 'd': > + dodebug = true; > + break; > + case 'v': > + fprintf(stdout, "%s\n", PACKAGE_VERSION); > + exit(EXIT_SUCCESS); > + break; > + case 'h': > + default: /* '?' */ > + fprintf(c == 'h' ? stdout : stderr, > + "Usage: %s [--debug|-d] [--help|-h] [--version|-v]\n", > + argv[0]); > + exit(c == 'h' ? EXIT_SUCCESS : EXIT_FAILURE); > + } > + } > + > + if (!dodebug) > + setenv("TSS2_LOG", "all+none", 1); > + > + cpu_sig (); > + > + exit(EXIT_SUCCESS); > +} > diff --git a/virt-what-cvm.pod b/virt-what-cvm.pod > new file mode 100644 > index 0000000..12cfc6a > --- /dev/null > +++ b/virt-what-cvm.pod > @@ -0,0 +1,195 @@ > +=encoding utf8 > + > +=head1 NAME > + > +virt-what-cvm - detect if we are running in a confidential virtual machine > + > +=head1 SUMMARY > + > +virt-what-cvm [options] > + > +=head1 DESCRIPTION > + > +C<virt-what-cvm> is a tool which can be used to detect if the program > +is running in a confidential virtual machine. > + > +The program prints out a list of "facts" about the confidential virtual > +machine, derived from heuristics. One fact is printed per line. > + > +If nothing is printed and the script exits with code 0 (no error), > +then it can mean I<either> that the program is running on bare-metal > +I<or> the program is running inside a non-confidential virtual machine, > +I<or> inside a type of confidential virtual machine which we don't know > +about or cannot detect. > + > +=head1 FACTS > + > +=over 4 > + > +=item B<amd-sev> > + > +This is a confidential guest running with AMD SEV technology > + > +Status: tested on Fedora 37 QEMU+KVM > + > +=item B<amd-sev-es> > + > +This is a confidential guest running with AMD SEV-ES technology > + > +Status: tested on Fedora 37 QEMU+KVM > + > +=item B<amd-sev-snp> > + > +This is a confidential guest running with AMD SEV-SNP technology > + > +Status: tested on Microsoft Azure SEV-SNP CVM > + > +Status: tested on Fedora 38 QEMU+KVM SEV-SNP (devel snapshot) > + > +=item B<intel-tdx> > + > +This is a confidential guest running with Intel TDX technology > + > +Status: tested on Microsoft Azure TDX CVM (preview) > + > +=item B<azure-hcl> > + > +This is a confidential guest running unenlightened under the > +Azure HCL (Host Compatibility Layer). This will be paired with > +B<amd-sev-snp>. > + > +Status: tested on Microsoft Azure SEV-SNP CVM > + > +=back > + > +=head1 EXIT STATUS > + > +Programs that use or wrap C<virt-what-cvm> should check that the exit > +status is 0 before they attempt to parse the output of the command. > + > +A non-zero exit status indicates some error, for example, an > +unrecognized command line argument. If the exit status is non-zero > +then the output "facts" (if any were printed) cannot be guaranteed and > +should be ignored. > + > +The exit status does I<not> have anything to do with whether the > +program is running on baremetal or under confidential virtualization, > +nor with whether C<virt-what-cvm> managed detection "correctly" (which > +is basically unknowable given the large variety of virtualization > +systems out there) > + > +=head1 RUNNING VIRT-WHAT-CVM FROM OTHER PROGRAMS > + > +C<virt-what-cvm> is designed so that you can easily run it from > +other programs or wrap it up in a library. > + > +Your program should check the exit status (see the section above). > + > +=head1 IMPORTANT NOTE > + > +This program detects whether it is likely to be running within a known > +confidential VM, but does I<NOT> prove that the environment is trustworthy. > +To attain trust in the environment requires an attestation report for the > +virtual machine, which is then verified by an already trusted 3rd party. > + > +The hardware features that this program relies on to establish facts > +about the confidential virtualization environment, are those features > +whose behaviour will be proved by verification of an attestation report. > + > +This program I<MAY> have false positives. ie it may report that it is a > +confidential VM when it is in fact a non-confidential VM faking it. > + > +This program I<SHOULD NOT> have false negatives. ie it should not fail to > +report existance of a confidential VM. Caveat that this only applies to > +environments which have been explicitly tested. > + > +If this program does print a fact, this can be used for enabling or > +disabling use of certain features, according to whether they are > +appropriate for a confidential environment. None the less, the VM > +I<MUST NOT> be trusted until an attestation report is verified. > + > +As a protection against false negatives from this tool, environments > +requiring high assurance should take one or more of these measures: > + > + * The facts reported by this program I<SHOULD> should be measured > + into one of the TPM PCRs > + * The attestation report I<SHOULD> cover the facts reported by > + this program > + * The attestation report I<SHOULD> should cover the enablement > + status of any features affected by decisions involving facts > + reported by this tool > + > +=head1 SEE ALSO > + > +L<http://people.redhat.com/~rjones/virt-what/>, > +L<https://github.com/Azure/confidential-computing-cvm-guest-attestation>, > +L<https://virtee.io/> > + > +=head1 AUTHORS > + > +Daniel P. Berrangé <berrange @ redhat . com> > + > +=head1 COPYRIGHT > + > +(C) Copyright 2023 Red Hat Inc., > +L<http://people.redhat.com/~rjones/virt-what/> > + > +This program is free software; you can redistribute it and/or modify > +it under the terms of the GNU General Public License as published by > +the Free Software Foundation; either version 2 of the License, or > +(at your option) any later version. > + > +This program is distributed in the hope that it will be useful, > +but WITHOUT ANY WARRANTY; without even the implied warranty of > +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +GNU General Public License for more details. > + > +You should have received a copy of the GNU General Public License > +along with this program; if not, write to the Free Software > +Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. > + > +=head1 REPORTING BUGS > + > +Bugs can be viewed on the Red Hat Bugzilla page: > +L<https://bugzilla.redhat.com/>. > + > +If you find a bug in virt-what-cvm, please follow these steps to report it: > + > +=over 4 > + > +=item 1. Check for existing bug reports > + > +Go to L<https://bugzilla.redhat.com/> and search for similar bugs. > +Someone may already have reported the same bug, and they may even > +have fixed it. > + > +=item 2. Capture debug and error messages > + > +Run > + > + virt-what-cvm -d > virt-what-cvm.log 2>&1 > + > +and keep I<virt-what-cvm.log>. It may contain error messages which you > +should submit with your bug report. > + > +=item 3. Get version of virt-what-cvm. > + > +Run > + > + virt-what-cvm --version > + > +=item 4. Submit a bug report. > + > +Go to L<https://bugzilla.redhat.com/> and enter a new bug. > +Please describe the problem in as much detail as possible. > + > +Remember to include the version numbers (step 3) and the debug > +messages file (step 2) and as much other detail as possible. > + > +=item 5. Assign the bug to rjones @ redhat.com > + > +Assign or reassign the bug to B<rjones @ redhat.com> (without the > +spaces). You can also send me an email with the bug number if you > +want a faster response. > + > +=back > -- > 2.40.1 -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW