A polite ping for comments on this version, thanks! On Tuesday, August 6, 2019 3:16 PM, Wei Wang wrote: > Last Branch Recording (LBR) is a performance monitor unit (PMU) feature on > Intel CPUs that captures branch related info. This patch series enables this > feature to KVM guests. > > Each guest can be configured to expose this LBR feature to the guest via > userspace setting the enabling param in KVM_CAP_X86_GUEST_LBR (patch > 3). > > About the lbr emulation method: > Since the vcpu get scheduled in, the lbr related msrs are made interceptible. > This makes guest first access to a lbr related msr always vm-exit to kvm, so > that kvm can know whether the lbr feature is used during the vcpu time slice. > The kvm lbr msr handler does the following > things: > - create an lbr perf event (task pinned) for the vcpu thread. > The perf event mainly serves 2 purposes: > -- follow the host perf scheduling rules to manage the vcpu's usage > of lbr (e.g. a cpu pinned lbr event could reclaim lbr and thus > stopping the vcpu's use); > -- have the host perf do context switching of the lbr state on the > vcpu thread switching. > - pass the lbr related msrs through to the guest. > This enables the following guest accesses to the lbr related msrs > without vm-exit, as long as the vcpu's lbr event owns the lbr feature. > A cpu pinned lbr event on the host could come and take over the lbr > feature via IPI calls. In this case, the pass-through will be > cancelled (patch 13), and the guest following accesses to the lbr msrs > will vm-exit to kvm and accesses will be forbidden in the handler. > > If the guest doesn't touch any of the lbr related msrs (likely the guest doesn't > need to run lbr in the near future), the vcpu's lbr perf event will be freed > (please see patch 12 commit for more details). > > * Tests > Conclusion: the profiling results on the guest are similar to that on the host. > > Run: ./perf -b ./test_program > > - Test on the host: > Overhead Command Source Shared Object Source Symbol Target > Symbol > 22.35% ftest libc-2.23.so [.] __random [.] > __random > 8.20% ftest ftest [.] qux [.] qux > 5.88% ftest ftest [.] random@plt [.] > __random > 5.88% ftest libc-2.23.so [.] __random [.] > __random_r > 5.79% ftest ftest [.] main [.] > random@plt > 5.60% ftest ftest [.] main [.] foo > 5.24% ftest libc-2.23.so [.] __random [.] main > 5.20% ftest libc-2.23.so [.] __random_r [.] > __random > 5.00% ftest ftest [.] foo [.] qux > 4.91% ftest ftest [.] main [.] bar > 4.83% ftest ftest [.] bar [.] qux > 4.57% ftest ftest [.] main [.] main > 4.38% ftest ftest [.] foo [.] main > 4.13% ftest ftest [.] qux [.] foo > 3.89% ftest ftest [.] qux [.] bar > 3.86% ftest ftest [.] bar [.] main > > - Test on the guest: > Overhead Command Source Shaged Object Source Symbol Target > Symbol > 22.36% ftest libc-2.23.so [.] random [.] random > 8.55% ftest ftest [.] qux [.] qux > 5.79% ftest libc-2.23.so [.] random [.] > random_r > 5.64% ftest ftest [.] random@plt [.] > random > 5.58% ftest ftest [.] main [.] > random@plt > 5.55% ftest ftest [.] main [.] foo > 5.41% ftest libc-2.23.so [.] random [.] main > 5.31% ftest libc-2.23.so [.] random_r [.] random > 5.11% ftest ftest [.] foo [.] qux > 4.93% ftest ftest [.] main [.] main > 4.59% ftest ftest [.] qux [.] bar > 4.49% ftest ftest [.] bar [.] main > 4.42% ftest ftest [.] bar [.] qux > 4.16% ftest ftest [.] main [.] bar > 3.95% ftest ftest [.] qux [.] foo > 3.79% ftest ftest [.] foo [.] main > (due to the lib version difference, "random" is equavlent to __random above) > > v7->v8 Changelog: > - Patch 3: > -- document KVM_CAP_X86_GUEST_LBR in api.txt > -- make the check of KVM_CAP_X86_GUEST_LBR return the size of > struct x86_perf_lbr_stack, to let userspace do a compatibility > check. > - Patch 7: > -- support perf scheduler to not assign a counter for the perf event > that has PERF_EV_CAP_NO_COUNTER set (rather than skipping the > perf > scheduler). This allows the scheduler to detect lbr usage conflicts > via get_event_constraints, and lower priority events will finally > fail to use lbr. > -- define X86_PMC_IDX_NA as "-1", which represents a never assigned > counter id. There are other places that use "-1", but could be > updated to use the new macro in another patch series. > - Patch 8: > -- move the event->owner assignment into perf_event_alloc to have it > set before event_init is called. Please see this patch's commit for > reasons. > - Patch 9: > -- use "exclude_host" and "is_kernel_event" to decide if the lbr event > is used for the vcpu lbr emulation, which doesn't need a counter, > and removes the usage of the previous new perf_event_create API. > -- remove the unused attr fields. > - Patch 10: > -- set a hardware reserved bit (bit 62 of LBR_SELECT) to reg->config > for the vcpu lbr emulation event. This makes the config different > from other host lbr event, so that they don't share the lbr. > Please see the comments in the patch for the reasons why they > shouldn't share. > - Patch 12: > -- disable interrupt and check if the vcpu lbr event owns the lbr > feature before kvm writing to the lbr related msr. This avoids kvm > updating the lbr msrs after lbr has been reclaimed by other events > via ipi. > -- remove arch v4 related support. > - Patch 13: > -- double check if the vcpu lbr event owns the lbr feature before > vm-entry into the guest. The lbr pass-through will be cancelled if > lbr feature has been reclaimed by a cpu pinned lbr event. > > Previous: > https://lkml.kernel.org/r/1562548999-37095-1-git-send-email-wei.w.wang > @intel.com > > Wei Wang (14): > perf/x86: fix the variable type of the lbr msrs > perf/x86: add a function to get the addresses of the lbr stack msrs > KVM/x86: KVM_CAP_X86_GUEST_LBR > KVM/x86: intel_pmu_lbr_enable > KVM/x86/vPMU: tweak kvm_pmu_get_msr > KVM/x86: expose MSR_IA32_PERF_CAPABILITIES to the guest > perf/x86: support to create a perf event without counter allocation > perf/core: set the event->owner before event_init > KVM/x86/vPMU: APIs to create/free lbr perf event for a vcpu thread > perf/x86/lbr: don't share lbr for the vcpu usage case > perf/x86: save/restore LBR_SELECT on vcpu switching > KVM/x86/lbr: lbr emulation > KVM/x86/vPMU: check the lbr feature before entering guest > KVM/x86: remove the common handling of the debugctl msr > > Documentation/virt/kvm/api.txt | 26 +++ > arch/x86/events/core.c | 36 ++- > arch/x86/events/intel/core.c | 3 + > arch/x86/events/intel/lbr.c | 95 +++++++- > arch/x86/events/perf_event.h | 6 +- > arch/x86/include/asm/kvm_host.h | 5 + > arch/x86/include/asm/perf_event.h | 17 ++ > arch/x86/kvm/cpuid.c | 2 +- > arch/x86/kvm/pmu.c | 24 +- > arch/x86/kvm/pmu.h | 11 +- > arch/x86/kvm/pmu_amd.c | 7 +- > arch/x86/kvm/vmx/pmu_intel.c | 476 > +++++++++++++++++++++++++++++++++++++- > arch/x86/kvm/vmx/vmx.c | 4 +- > arch/x86/kvm/vmx/vmx.h | 2 + > arch/x86/kvm/x86.c | 47 ++-- > include/linux/perf_event.h | 18 ++ > include/uapi/linux/kvm.h | 1 + > kernel/events/core.c | 19 +- > 18 files changed, 738 insertions(+), 61 deletions(-) > > -- > 2.7.4
