On Fri Dec 1, 2023 at 5:47 PM UTC, Sean Christopherson wrote: > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. > > > > On Fri, Dec 01, 2023, Nicolas Saenz Julienne wrote: > > On Fri Dec 1, 2023 at 4:32 PM UTC, Sean Christopherson wrote: > > > On Fri, Dec 01, 2023, Nicolas Saenz Julienne wrote: > > > > > To support this I think that we can add a userspace msr filter on the HV_X64_MSR_HYPERCALL, > > > > > although I am not 100% sure if a userspace msr filter overrides the in-kernel msr handling. > > > > > > > > I thought about it at the time. It's not that simple though, we should > > > > still let KVM set the hypercall bytecode, and other quirks like the Xen > > > > one. > > > > > > Yeah, that Xen quirk is quite the killer. > > > > > > Can you provide pseudo-assembly for what the final page is supposed to look like? > > > I'm struggling mightily to understand what this is actually trying to do. > > > > I'll make it as simple as possible (diregard 32bit support and that xen > > exists): > > > > vmcall <- Offset 0, regular Hyper-V hypercalls enter here > > ret > > mov rax,rcx <- VTL call hypercall enters here > > I'm missing who/what defines "here" though. What generates the CALL that points > at this exact offset? If the exact offset is dictated in the TLFS, then aren't > we screwed with the whole Xen quirk, which inserts 5 bytes before that first VMCALL? Yes, sorry, I should've included some more context. Here's a rundown (from memory) of how the first VTL call happens: - CPU0 start running at VTL0. - Hyper-V enables VTL1 on the partition. - Hyper-V enabled VTL1 on CPU0, but doesn't yet switch to it. It passes the initial VTL1 CPU state alongside the enablement hypercall arguments. - Hyper-V sets the Hypercall page overlay address through HV_X64_MSR_HYPERCALL. KVM fills it. - Hyper-V gets the VTL-call and VTL-return offset into the hypercall page using the VP Register HvRegisterVsmCodePageOffsets (VP register handling is in user-space). - Hyper-V performs the first VTL-call, and has all it needs to move between VTL0/1. Nicolas