On 2023-01-31 at 08:28 -05, James Bottomley <jejb@xxxxxxxxxxxxx> wrote... > On Tue, 2023-01-31 at 11:31 +0000, Reshetova, Elena wrote: >> > On Mon, 2023-01-30 at 07:42 +0000, Reshetova, Elena wrote: >> > [...] >> > > > The big threat from most devices (including the thunderbolt >> > > > classes) is that they can DMA all over memory. However, this >> > > > isn't really a threat in CC (well until PCI becomes able to do >> > > > encrypted DMA) because the device has specific unencrypted >> > > > buffers set aside for the expected DMA. If it writes outside >> > > > that CC integrity will detect it and if it reads outside that >> > > > it gets unintelligible ciphertext. So we're left with the >> > > > device trying to trick secrets out of us by returning >> > > > unexpected data. >> > > >> > > Yes, by supplying the input that hasn’t been expected. This is >> > > exactly the case we were trying to fix here for example: >> > > https://lore.kernel.org/all/20230119170633.40944-2- >> > alexander.shishkin@xxxxxxxxxxxxxxx/ >> > > I do agree that this case is less severe when others where memory >> > > corruption/buffer overrun can happen, like here: >> > > https://lore.kernel.org/all/20230119135721.83345-6- >> > alexander.shishkin@xxxxxxxxxxxxxxx/ >> > > But we are trying to fix all issues we see now (prioritizing the >> > > second ones though). >> > >> > I don't see how MSI table sizing is a bug in the category we've >> > defined. The very text of the changelog says "resulting in a >> > kernel page fault in pci_write_msg_msix()." which is a crash, >> > which I thought we were agreeing was out of scope for CC attacks? >> >> As I said this is an example of a crash and on the first look >> might not lead to the exploitable condition (albeit attackers are >> creative). But we noticed this one while fuzzing and it was common >> enough that prevented fuzzer going deeper into the virtio devices >> driver fuzzing. The core PCI/MSI doesn’t seem to have that many >> easily triggerable Other examples in virtio patchset are more severe. > > You cited this as your example. I'm pointing out it seems to be an > event of the class we've agreed not to consider because it's an oops > not an exploit. If there are examples of fixing actual exploits to CC > VMs, what are they? > > This patch is, however, an example of the problem everyone else on the > thread is complaining about: a patch which adds an unnecessary check to > the MSI subsystem; unnecessary because it doesn't fix a CC exploit and > in the real world the tables are correct (or the manufacturer is > quickly chastened), so it adds overhead to no benefit. I'd like to backtrack a little here. 1/ PCI-as-a-thread, where does it come from? On physical devices, we have to assume that the device is working. As other pointed out, there are things like PCI compliance tests, etc. So Linux has to trust the device. You could manufacture a broken device intentionally, but the value you would get from that would be limited. On a CC system, the "PCI" values are really provided by the hypervisor, which is not trusted. This leads to this peculiar way of thinking where we say "what happens if virtual device feeds us a bogus value *intentionally*". We cannot assume that the *virtual* PCI device ran through the compliance tests. Instead, we see the PCI interface as hostile, which makes us look like weirdos to the rest of the community. Consequently, as James pointed out, we first need to focus on consequences that would break what I would call the "CC promise", which is essentially that we'd rather kill the guest than reveal its secrets. Unless you have a credible path to a secret being revealed, don't bother "fixing" a bug. And as was pointed out elsewhere in this thread, caching has a cost, so you can't really use the "optimization" angle either. 2/ Clarification of the "CC promise" and value proposition Based on the above, the very first thing is to clarify that "CC promise", because if exchanges on this thread have proved anything, it is that it's quite unclear to anyone outside the "CoCo world". The Linux Guest Kernel Security Specification needs to really elaborate on what the value proposition of CC is, not assume it is a given. "Bug fixes" before this value proposition has been understood and accepted by the non-CoCo community are likely to go absolutely nowhere. Here is a quick proposal for the Purpose and Scope section: <doc> Purpose and Scope Confidential Computing (CC) is a set of technologies that allows a guest to run without having to trust either the hypervisor or the host. CC offers two new guarantees to the guest compared to the non-CC case: a) The guest will be able to measure and attest, by cryptographic means, the guest software stack that it is running, and be assured that this software stack cannot be tampered with by the host or the hypervisor after it was measured. The root of trust for this aspect of CC is typically the CPU manufacturer (e.g. through a private key that can be used to respond to cryptographic challenges). b) Guest state, including memory, become secrets which must remain inaccessible to the host. In a CC context, it is considered preferable to stop or kill a guest rather than risk leaking its secrets. This aspect of CC is typically enforced by means such as memory encryption and new semantics for memory protection. CC leads to a different threat model for a Linux kernel running as a guest inside a confidential virtual machine (CVM). Notably, whereas the machine (CPU, I/O devices, etc) is usually considered as trustworthy, in the CC case, the hypervisor emulating some aspects of the virtual machine is now considered as potentially malicious. Consequently, effects of any data provided by the guest to the hypervisor, including ACPI configuration tables, MMIO interfaces or machine specific registers (MSRs) need to be re-evaluated. This document describes the security architecture of the Linux guest kernel running inside a CVM, with a particular focus on the Intel TDX implementation. Many aspects of this document will be applicable to other CC implementations such as AMD SEV. Aspects of the guest-visible state that are under direct control of the hardware, such as the CPU state or memory protection, will be considered as being handled by the CC implementations. This document will therefore only focus on aspects of the virtual machine that are typically managed by the hypervisor or the host. Since the host ultimately owns the resources and can allocate them at will, including denying their use at any point, this document will not address denial or service or performance degradation. It will however cover random number generation, which is central for cryptographic security. Finally, security considerations that apply irrespective of whether the platform is confidential or not are also outside of the scope of this document. This includes topics ranging from timing attacks to social engineering. </doc> Feel free to comment and reword at will ;-) 3/ PCI-as-a-threat: where does that come from Isn't there a fundamental difference, from a threat model perspective, between a bad actor, say a rogue sysadmin dumping the guest memory (which CC should defeat) and compromised software feeding us bad data? I think there is: at leats inside the TCB, we can detect bad software using measurements, and prevent it from running using attestation. In other words, we first check what we will run, then we run it. The security there is that we know what we are running. The trust we have in the software is from testing, reviewing or using it. This relies on a key aspect provided by TDX and SEV, which is that the software being measured is largely tamper-resistant thanks to memory encryption. In other words, after you have measured your guest software stack, the host or hypervisor cannot willy-nilly change it. So this brings me to the next question: is there any way we could offer the same kind of service for KVM and qemu? The measurement part seems relatively easy. Thetamper-resistant part, on the other hand, seems quite difficult to me. But maybe someone else will have a brilliant idea? So I'm asking the question, because if you could somehow prove to the guest not only that it's running the right guest stack (as we can do today) but also a known host/KVM/hypervisor stack, we would also switch the potential issues with PCI, MSRs and the like from "malicious" to merely "bogus", and this is something which is evidently easier to deal with. I briefly discussed this with James, and he pointed out two interesting aspects of that question: 1/ In the CC world, we don't really care about *virtual* PCI devices. We care about either virtio devices, or physical ones being passed through to the guest. Let's assume physical ones can be trusted, see above. That leaves virtio devices. How much damage can a malicious virtio device do to the guest kernel, and can this lead to secrets being leaked? 2/ He was not as negative as I anticipated on the possibility of somehow being able to prevent tampering of the guest. One example he mentioned is a research paper [1] about running the hypervisor itself inside an "outer" TCB, using VMPLs on AMD. Maybe something similar can be achieved with TDX using secure enclaves or some other mechanism? Sorry, this mail is a bit long ;-) > > > [...] >> > see what else it could detect given the signal will be smothered by >> > oopses and secondly I think the PCI interface is likely the wrong >> > place to begin and you should probably begin on the virtio bus and >> > the hypervisor generated configuration space. >> >> This is exactly what we do. We don’t fuzz from the PCI config space, >> we supply inputs from the host/vmm via the legitimate interfaces that >> it can inject them to the guest: whenever guest requests a pci config >> space (which is controlled by host/hypervisor as you said) read >> operation, it gets input injected by the kafl fuzzer. Same for other >> interfaces that are under control of host/VMM (MSRs, port IO, MMIO, >> anything that goes via #VE handler in our case). When it comes to >> virtio, we employ two different fuzzing techniques: directly >> injecting kafl fuzz input when virtio core or virtio drivers gets the >> data received from the host (via injecting input in functions >> virtio16/32/64_to_cpu and others) and directly fuzzing DMA memory >> pages using kfx fuzzer. More information can be found in >> https://intel.github.io/ccc-linux-guest-hardening-docs/tdx-guest-hardening.html#td-guest-fuzzing > > Given that we previously agreed that oppses and other DoS attacks are > out of scope for CC, I really don't think fuzzing, which primarily > finds oopses, is at all a useful tool unless you filter the results by > the question "could we exploit this in a CC VM to reveal secrets". > Without applying that filter you're sending a load of patches which > don't really do much to reduce the CC attack surface and which do annoy > non-CC people because they add pointless checks to things they expect > the cards and config tables to get right. Indeed. [1]: https://dl.acm.org/doi/abs/10.1145/3548606.3560592 -- Cheers, Christophe de Dinechin (https://c3d.github.io) Theory of Incomplete Measurements (https://c3d.github.io/TIM)
