Wiki - https://fedoraproject.org/wiki/Changes/IntelSGX Discussion Thread - https://discussion.fedoraproject.org/t/f42-change-proposal-intel-sgx-software-stack-self-contained/138981 This is a proposed Change for Fedora Linux. This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == The Intel SGX technology enables creation of execution enclaves, whose memory is encrypted and thus protected from all other code running on the CPU, including SMM, firmware, kernel and userspace. This proposal is to introduce the SGX host software stack, architectural enclaves and development packages to Fedora, to enable future introduction applications and features which have a dependency on SGX technology. The primary feature that will leverage SGX in a subsequent Fedora release is expected to be Intel TDX, which provides confidential virtual machines, and is in the process of being integrated with QEMU and Linux/KVM. == Owner == * Name: [[User:berrange| Daniel Berrange]] * Email: berrange@xxxxxxxxxx == Detailed Description == The Intel SGX technology enables creation of execution enclaves, whose memory is encrypted and thus protected from all other code running on the machine, including SMM, firmware, kernel and userspace. While it has many potential use cases, this proposal is focused around the infrastructure needed to enable support for attestation of TDX confidential virtual machines. The SGX software stack that is to be introduced can be categorized into a number of distinct areas: * Support for developing new enclaves ** Header files for the enclave code (a minimalist C library, C++ library, crypto and some other misc libraries) ** Static library archives for linking into the enclave binaries. ** Build helper tools (for signing enclaves, generating code enclave API entrypoints) * Support for developing applications that use enclaves, eg to be able to load and communicate with enclaves. ** Header files for platform code ** Dynamic libraries for platform code * Support for deploying applications that use enclaves ** Enclave service daemon - assists unprivileged applications in loading enclaves ** Registration tools - assists platform administrator in acquiring certificates from Intel servers to identity the platform ** Quote generation daemon - assists QEMU in acquiring signed attestation reports for TDX VMs. The code and binaries related to the host OS platform components will be installed in the normal filesystem locations common to all Fedora packages. e.g. libraries in '''/usr/lib64''', headers in '''/usr/include''' and binaries in '''/usr/bin'''. For the purposes of packaging, the enclaves will be [https://fedoraproject.org/wiki/Virt/SGXEnclaves#Enclaves_as_a_platform_build_target treated as cross-compilation target]. While the compiler build architecture target is '''x86_64''', the runtime has custom C / C++ libraries that must be used, and requires a separate code loader, all completely separate from any existing Fedora libraries. Enclaves cannot be directly linked to applications, they are strictly independent & self-contained ELF binaries. With this in mind, following the example of MinGW64 packages living under '''/usr/x86_64-w64-mingw''', all enclave related headers and libraries are proposed to be placed under '''/usr/x86_64-intel-sgx''', specifically the '''lib64''' and '''include''' sub-directories. There is no concept of executable binaries for enclaves, only libraries, nor ancillary data files. Thus no '''bin''' or '''share''' directories are required under this location. The installation tree locations will be defined in RPM macros provided by the '''sgx-srpm-macros''' package The generated binary packages will generally all have an ''sgx-'' prefix to their name, with those containing exclusively enclave related content having the more specialized ''sgx-enclave-'' name prefix. For reasons of brevity, this change proposal does not go far into all technical details of SGX. For a deeper understanding of SGX enclaves beyond this proposal, [https://fedoraproject.org/wiki/Virt/SGXEnclaves consult this companion document]. == Feedback == * '''Feedback:''' ''The SGX technology can be used as a way to implement DRM. Notable example has been BluRay playback.''. '''Answer:''' As with many technologies, it is possible to use SGX in ways that are both positive and negative, wrt the owner / users of a machine. Use of SGX for DRM in BluRay playback is hostile to the owner/user of a machine. This change is '''NOT''' proposing to introduce / support any such usage/applications in Fedora. The fact that bad uses of SGX may exist outside of Fedora, is no justification for rejecting the use of SGX in Fedora when applied to scenarios that can offer features directly benefiting Fedora's users. * '''Feedback:''' ''The SGX enclave code is not open source, because it requires a vendor signature on output binaries''. '''Answer:''' All of the SGX code is made available under a variety open source licenses (Apache, BSD, MIT, GPL & more), which are fully approved by Fedora. One of the architectural enclaves, '''pce''', requires an Intel signature because it is used to establish the root of trust with the hardware. In their role of bootstrapping use of SGX hardware, the architectural enclaves are a type of firmware, and it is normal for firmware to have a vendor signature. In contrast to almost all firmware which is proprietary, the architectural enclave code is all under Fedora approved OSS licenses. All of the architectural enclaves, can be rebuilt from source by a user, with customizations, signed with a user specified key and then loaded if desired. There are some subtle limitations if attempting this, documented in [[Virt/SGXEnclaves#Rebuilding_&_self-signing_architectural_enclaves | the companion document]], but the code is none the less all OSS. == Benefit to Fedora == As a general purpose infrastructure technology, SGX can be applied to / used by a wide variety of scenarios / applications. In the context of this change proposal, '''no application''' usage is intended to be introduced, the focus is around general software infrastructure enablement. A followup change proposal (for a subsequent Fedora release) will be made to introduce Intel TDX confidential virtual machines, which is anticipated to be the first end user facing application of SGX technology. Attestation is the means by which a guest VM owner can prove that their VM machine is running in confidential mode on genuine Intel hardware, as opposed to in a faked "blue pill" environment. All currently shipping Intel CPUs which support TDX build attestation on top of SGX with OSS enclave code (the '''tdqe''' enclave), as opposed to embedding attestation in proprietary firmware which is the approach taken by other vendors for confidential computing. NB, the '''tdqe''' enclave and '''qgs''' daemon will be included in this proposal, but will remain unused until this followup TDX support is integrated with Fedora. It is well known that SGX technology can be applied is to build DRM systems which allow 3rd party organizations to control what users do with their machines. This proposal '''MUST NOT''' be interpreted as endorsing such use cases. Such DRM systems would almost certainly rely on close source enclaves signed exclusively by a 3rd party org, and could not be considered hardware enablement firmware. As such they would not be eligible for inclusion in Fedora. == Scope == === Proposal owners=== The proposal owners will be introducing a number of new packages to Fedora. Initially, no existing packages are likely to gain dependencies on the new SGX packages. At a future date when TDX is integrated into Fedora, one or more of the libvirt/qemu sub-RPMs may gain a dependency on certain SGX packages needed to support TDX. The important new packages will be * '''sgx-srpm-macros''': this will produce a binary RPM of the same name, containing RPM macros that define the directory locations for SGX enclaves under '''/usr/x86_64-intel-sgx'''. These macros will be consumed by other RPMs that deal with enclave pacakging * '''linux-sgx''': this provides the full SGX software stack, including the enclave development libraries and headers, host OS development libraries and headers, supporting host OS build tools, and various host OS runtime daemons/binaries. This package will include '''unsigned''' builds of the architectural enclaves built using the standard Fedora toolchain. Users could choose to sign these with their own keys and deploy them, with the caveat noted earlier wrt the '''pce''' enclave in particular. This package will result in creation of many binary sub-RPMs for distinct components. * '''linux-sgx-enclaves-prebuilt''': this provides solely the pre-built, signed architectural enclaves. While they could be provided as part of the '''linux-sgx''' package, it is split off to make it simpler to update the architectural enclave independently of the main software stack. This ability will be important if needing to ship a new release of SGX enclaves for a security fix in stable Fedora branches, without being forced to introduce updates of the host OS software stack that may have inappropriate changes for pushing into an existing stable branch. ==== Optional extra: reproducible builds ==== The SGX pre-built, signed architectural enclaves are built from code entirely under Fedora approved OSS licenses and support a reproducible build process. The upstream implementation of this process involves an Ubuntu dockerfile that populates a NixOS build environment. With some skilled work, it is possible to replicate an equivalent build environment in the context of Fedora. It requires providing custom builds of GCC, binutils, nasm, glibc (headers only) and kernel (uapi headers only) at specific release versions, with specific build configuration options and certain patches. Given such packages in Fedora, it is then possible to perform a reproducible build that successfully validates as matching the code payload of the pre-built, signed architectural enclaves. Maintaining such a reproducible build process in Fedora would have a non-negligible maintenance overhead if done on an ongoing basis. Originally it was thought that a reproducible build could be performed in the same package that provides the pre-built enclave binaries, as a way to block shipping of the pre-built binaries, unless proved to match the source. Given the requirements for specific toolchain versions, however, this would impose considerable delays when shipping CVE updates in stable Fedora branches. eg a newly issued pre-built enclave binary containing a CVE fix, may imply use of newer GCC, binutils, nasm versions than are currently available in the Fedora stable release. It is highly undesirable to go through the new package process to add new toolschain versions to stable Fedora, as a pre-requisite to ship CVE fixes in a stable branch, even if the new package process were expedited to avoid full re-review. If a reproducible build were to be mandated, it would involve the following additional packages in Fedora: * '''sgx-compat-gccXXXX''' - one (or more) GCC builds at specified designated versions required by the reproducible build * '''sgx-compat-binutilsXXXX''' - one (or more) binutils builds at specified designated versions required by the reproducible build * '''sgx-compat-nasmXXXX''' - one NASM build at specified designated versions required by the reproducible build * '''sgx-compat-glibc-headersXXXX''' - one glibc build at specified designated versions required by the reproducible build. Note only the header files are required, no library build. * '''sgx-compat-kernel-headersXXXX''' - one kernel build at specified designated versions required by the reproducible build. Note only the uapi header files are required, no vmlinux build * '''sgx-enclaves-reproducibleXXX''' - one (or more) reproducible builds of SGX architectural enclaves Note, the need for multiple versions of GCC & binutils in the reproducible build is an [https://github.com/intel/linux-sgx/issues/1045 upstream build environment bug]. Note, the need for glibc & header headers is also a likely [https://github.com/intel/linux-sgx/pull/1062 upstream build environment bug]. Note, the need for multiple '''sgx-enclaves-reproducibleXXX''' versions in parallel is because not all enclaves re-issued on each release. Only enclaves which have CVE fixes or functional changes get re-issued as new pre-built, signed binaries. IOW to validate all binary enclaves potentially requires doing reproducible builds of multiple SGX releases. === Other developers=== Maintainers of the '''fedora-release''' package will need to review a merge request that proposes new systemd unit file presets. See later in this document for the details of units. ===Release engineering=== N/A - does not impact deliverables for releng ===Policies and guidelines=== Agreement is needed around the designation of the pre-built, signed SGX architectural enclaves as firmware. Note, "'''architectural enclaves'''" are being considered distinct from "'''application enclaves'''" - the latter should be expected to build from source in the normal manner & be signed by Fedora, if any such enclaves are ever added to Fedora. ==== Architectural enclaves as firmware ==== Normal Fedora practice requires building everything from source. There is a general exception to this for firmware blobs, which don't need to be built from source and don't need to be under an OSS compliant license as long as the binaries can be freely distributed. Hardware firmware binaries almost always include a digital signature from the vendor, to ensure that only trusted firmware is loaded onto the device. Thus even if source code is available for a given firmware binary, Fedora would still need to be shipping it as a pre-built vendor supplied signed binary, in order to pass the cryptographic verification checks typically performed by hardware when loading firmware. Another example of shipping pre-built vendor binaries would be CPU microcode. This could be considered firmware for the core CPU, and again are vendor supplied, signed binaries, with no source code available. The architectural enclaves satisfy the classical role of firmware, in the context of the SGX technology, providing enablement for application usage. Unusually for firmware they are made available with 100% of their code under Fedora approved OSS licenses. The architectural enclaves, however, are still consumed as pre-built signed binaries due to the need to establish a chain of trust rooted back to Intel as the CPU hardware vendor. '''Overall''', the architectural enclaves meet the definition of firmware documented in the [https://docs.fedoraproject.org/en-US/legal/license-approval/#_technical_firmware_requirements Fedora licensing guidelines]: # ''The files must be non-executable within the Fedora Linux context (note: this means that the files cannot run on their own, not that they are just chmod -x).'' ✅ enclave ELF binarjies must be loaded into memory using the specialized SGX loader. # ''The files must not be libraries, within the Fedora Linux context.'' ✅ enclave ELF binaries must be loaded into memory using as specialized SGX loader, the glibc ELF loader can not be used for this. # ''The files must be standalone, not embedded in executable or library code (within the Fedora Linux context).'' ✅ enclave binaries are always standalone, self-contained files. They are consumed by regular host software, but cannot be linked to Fedora platform binaries/libraries. # ''The files must be necessary for the functionality of open source code being included in Fedora Linux or to enable Fedora Linux to boot on a specific device, where no other reliable and supported mechanisms exist.'' ✅ the pre-built, signed '''architectural enclaves''' shipped by Intel, serve to bootstrap the use of SGX technology by applications. ❌ as higher level software services, other '''application enclaves''' would not qualify. Thus it should be permissible to ship the pre-built, signed '''architectural enclaves''' binaries with no policy changes. Conversely '''application enclaves''' should NOT be considered firmware, and thus be required to follow the full Fedora packaging guidelines. No '''application enclaves''' are anticipated for inclusion in Fedora with this change proposal. ==== Reproducible build validation policy ==== Even if the view of '''architectural enclaves''' as firmware were to be rejected, there is an alternative precedent that can justify inclusion of the pre-built signed enclaves in Fedora. In this case we consider the '''shim''' package, where Fedora builds a binary from known good source, possibly with local patches, and sends it off for signing by Microsoft, packaging the binary that is sent back into an RPM. Microsoft has various requirements before it will permit signing a vendor's shim. Consider if the requirements on shim signing were tightened a little more to require a designated toolchain version, such that all shim builds became 100% reproducible on any platform. There would be no need to send the binary to Microsoft for signing, as any binary built in Fedora would inherently match a standard pre-published signed binary from that shim release + toolchain. It would be impossible to distinguish Microsoft signing & returning Fedora's reproducible build, from Microsoft sending back their original pre-signed build. This is effectively the situation SGX architectural enclaves are in. None the less, there are some differences between the shim & SGX scenarios. The first is that today Fedora has the ability to add arbitrary local patches to shim & build with arbitrary toolchain versions, which would be in conflict with a hypothetical requirement for reproducible builds & fixed toolchain. The second is that on most (but not all) hardware, users can enroll personal certificates in the firmware to allow it to trust a shim binary signed by the user, rather than Microsoft. In the case of SGX enclaves, it is strictly possible from a technical POV for a user to build & load & run their own self-signed SGX architectural enclaves.
