On Fri, Sep 29, 2023, Peter Zijlstra wrote: > On Wed, Sep 27, 2023 at 10:27:07AM -0700, Sean Christopherson wrote: > > Jumping the gun a bit (we're in the *super* early stages of scraping together a > > rough PoC), but I think we should effectively put KVM's current vPMU support into > > maintenance-only mode, i.e. stop adding new features unless they are *very* simple > > to enable, and instead pursue an implementation that (a) lets userspace (and/or > > the kernel builder) completely disable host perf (or possibly just host perf usage > > of the hardware PMU) and (b) let KVM passthrough the entire hardware PMU when it > > has been turned off in the host. > > I don't think you need to go that far, host can use PMU just fine as > long as it doesn't overlap with a vCPU. Basically, if you force > perf_attr::exclude_guest on everything your vCPU can haz the full thing. Complexity aside, my understanding is that the overhead of trapping and emulating all of the guest counter and MSR accesses results in unacceptably degraded functionality for the guest. And we haven't even gotten to things like arch LBRs where context switching MSRs between the guest and host is going to be quite costly. > > Note, a similar idea was floated and rejected in the past[*], but that failed > > proposal tried to retain host perf+PMU functionality by making the behavior dynamic, > > which I agree would create an awful ABI for the host. If we make the "knob" a > > Kconfig > > Must not be Kconfig, distros would have no sane choice. Or not only a Kconfig? E.g. similar to how the kernel has CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS and nopku. > > or kernel param, i.e. require the platform owner to opt-out of using perf > > no later than at boot time, then I think we can provide a sane ABI, keep the > > implementation simple, all without breaking existing users that utilize perf in > > the host to profile guests. > > It's a shit choice to have to make. At the same time I'm not sure I have > a better proposal. > > It does mean a host cannot profile one guest and have pass-through on the > other. Eg. have a development and production guest on the same box. This > is pretty crap. > > Making it a guest-boot-option would allow that, but then the host gets > complicated again. I think I can make it trivially work for per-task > events, simply error the creation of events without exclude_guest for > affected vCPU tasks. But the CPU events are tricky. > > > I will firmly reject anything that takes the PMU away from the host > entirely through. Why? What is so wrong with supporting use cases where the platform owner *wants* to give up host PMU and NMI watchdog functionality? If disabling host PMU usage were complex, highly invasive, and/or difficult to maintain, then I can understand the pushback. But if we simply allow hiding hardware PMU support, then isn't the cost to perf just a few lines in init_hw_perf_events()? And if we put a stake in the ground and say that exposing "advanced" PMU features to KVM guests requires a passthrough PMU, i.e. the PMU to be hidden from the host, that will significantly reduce our maintenance and complexity. The kernel allows disabling almost literally every other feature that is even remotely optional, I don't understand why the hardware PMU is special.
