Hello KVM, MM and memfd_secret folks, Currently when using anonymous memory for KVM guest RAM, the memory all remains mapped into the kernel direct map. We are looking at options to get KVM guest memory out of the kernel’s direct map as a principled approach to mitigating speculative execution issues in the host kernel. Our goal is to more completely address the class of issues whose leak origin is categorized as "Mapped memory" [1]. We currently have downstream-only solutions to this, but we want to move to purely upstream code. So far we have been looking at using memfd_secret, which seems to be designed exactly for usecases where it is undesirable to have some memory range accessible through the kernel’s direct map. However, memfd_secret doesn’t work out the box for KVM guest memory; the main reason seems to be that the GUP path is intentionally disabled for memfd_secret, so if we use a memfd_secret backed VMA for a memslot then KVM is not able to fault the memory in. If it’s been pre-faulted in by userspace then it seems to work. There are a few other issues around when KVM accesses the guest memory. For example the KVM PV clock code goes directly to the PFN via the pfncache, and that also breaks if the PFN is not in the direct map, so we’d need to change that sort of thing, perhaps going via userspace addresses. If we remove the memfd_secret check from the GUP path, and disable KVM’s pvclock from userspace via KVM_CPUID_FEATURES, we are able to boot a simple Linux initrd using a Firecracker VMM modified to use memfd_secret. We are also aware of ongoing work on guest_memfd. The current implementation unmaps guest memory from VMM address space, but leaves it in the kernel’s direct map. We’re not looking at unmapping from VMM userspace yet; we still need guest RAM there for PV drivers like virtio to continue to work. So KVM’s gmem doesn’t seem like the right solution? With this in mind, what’s the best way to solve getting guest RAM out of the direct map? Is memfd_secret integration with KVM the way to go, or should we build a solution on top of guest_memfd, for example via some flag that causes it to leave memory in the host userspace’s page tables, but removes it from the direct map? We are keen to help contribute to getting this working, we’re just looking for guidance from maintainers on what the correct way to solve this is. Cheers, James + colleagues Derek and Patrick
