+Jim, David, and Mingwei On Wed, Sep 27, 2023, Peter Zijlstra wrote: > On Wed, Sep 27, 2023 at 11:31:18AM +0800, Dapeng Mi wrote: > > When guest wants to use PERF_METRICS MSR, a virtual metrics event needs > > to be created in the perf subsystem so that the guest can have exclusive > > ownership of the PERF_METRICS MSR. > > Urgh, can someone please remind me how all that is supposed to work > again? The guest is just a task that wants the event. If the > host creates a CPU event, then that gets scheduled with higher priority > and the task looses out, no joy. > > So you cannot guarantee the guest gets anything. > > That is, I remember we've had this exact problem before, but I keep > forgetting how this all is supposed to work. I don't use this virt stuff > (and every time I try qemu arguments defeat me and I give up in > disgust). I don't think it does work, at least not without a very, very carefully crafted setup and a host userspace that knows it must not use certain aspects of perf. E.g. for PEBS, if the guest virtual counters don't map 1:1 to the "real" counters in hardware, KVM+perf simply disables the counter. And for top-down slots, getting anything remotely accurate requires pinning vCPUs 1:1 with pCPUs and enumerating an accurate toplogy to the guest: The count is distributed among unhalted logical processors (hyper-threads) who share the same physical core, in processors that support Intel Hyper-Threading Technology. Jumping the gun a bit (we're in the *super* early stages of scraping together a rough PoC), but I think we should effectively put KVM's current vPMU support into maintenance-only mode, i.e. stop adding new features unless they are *very* simple to enable, and instead pursue an implementation that (a) lets userspace (and/or the kernel builder) completely disable host perf (or possibly just host perf usage of the hardware PMU) and (b) let KVM passthrough the entire hardware PMU when it has been turned off in the host. I.e. keep KVM's existing best-offset vPMU support, e.g. for setups where the platform owner is also the VM ueer (running a Windows VM on a Linux box, hosting a Linux VM in ChromeOS, etc...). But for anything advanced and for hard guarantees, e.g. cloud providers that want to expose fully featured vPMU to customers, force the platform owner to choose between using perf (or again, perf with hardware PMU) in the host, and exposing the hardware PMU to the guest. Hardware vendors are pushing us in the direction whether we like it or not, e.g. SNP and TDX want to disallow profiling the guest from the host, ARM has an upcoming PMU model where (IIUC) it can't be virtualized without a passthrough approach, Intel's hybrid CPUs are a complete trainwreck unless vCPUs are pinned, and virtualizing things like top-down slots, PEBS, and LBRs in the shared model requires an absurd amount of complexity throughout the kernel and userspace. Note, a similar idea was floated and rejected in the past[*], but that failed proposal tried to retain host perf+PMU functionality by making the behavior dynamic, which I agree would create an awful ABI for the host. If we make the "knob" a Kconfig or kernel param, i.e. require the platform owner to opt-out of using perf no later than at boot time, then I think we can provide a sane ABI, keep the implementation simple, all without breaking existing users that utilize perf in the host to profile guests. [*] https://lore.kernel.org/all/CALMp9eRBOmwz=mspp0m5Q093K3rMUeAsF3vEL39MGV5Br9wEQQ@xxxxxxxxxxxxxx
