Hi Alex, On Mon, Dec 13, 2021 at 3:14 AM Alexandru Elisei <alexandru.elisei@xxxxxxx> wrote: > > Hi Reiji, > > On Sun, Dec 12, 2021 at 10:36:52PM -0800, Reiji Watanabe wrote: > > On Wed, Dec 8, 2021 at 12:05 AM Marc Zyngier <maz@xxxxxxxxxx> wrote: > > > > > > Reji, > > > > > > On 2021-12-08 02:36, Reiji Watanabe wrote: > > > > Hi Alex, > > > > > > > > On Mon, Dec 6, 2021 at 9:02 AM Alexandru Elisei > > > > <alexandru.elisei@xxxxxxx> wrote: > > > >> > > > >> (CC'ing Peter Maydell in case this might be of interest to qemu) > > > >> > > > >> The series can be found on a branch at [1], and the kvmtool support at > > > >> [2]. > > > >> The kvmtool patches are also on the mailing list [3] and haven't > > > >> changed > > > >> since v1. > > > >> > > > >> Detailed explanation of the issue and symptoms that the patches > > > >> attempt to > > > >> correct can be found in the cover letter for v1 [4]. > > > >> > > > >> A brief summary of the problem is that on heterogeneous systems KVM > > > >> will > > > >> always use the same PMU for creating the VCPU events for *all* VCPUs > > > >> regardless of the physical CPU on which the VCPU is running, leading > > > >> to > > > >> events suddenly stopping and resuming in the guest as the VCPU thread > > > >> gets > > > >> migrated across different CPUs. > > > >> > > > >> This series proposes to fix this behaviour by allowing the user to > > > >> specify > > > >> which physical PMU is used when creating the VCPU events needed for > > > >> guest > > > >> PMU emulation. When the PMU is set, KVM will refuse to the VCPU on a > > > >> physical which is not part of the supported CPUs for the specified > > > >> PMU. > > > > > > > > Just to confirm, this series provides an API for userspace to request > > > > KVM to detect a wrong affinity setting due to a userspace bug so that > > > > userspace can get an error at KVM_RUN instead of leading to events > > > > suddenly stopping, correct ? > > > > > > More than that, it allows userspace to select which PMU will be used > > > for their guest. The affinity setting is a byproduct of the PMU's own > > > affinity. > > > > Thank you for the clarification. > > (I overlooked the change in kvm_pmu_create_perf_event()...) > > > > > > > > > > > >> The default behaviour stays the same - without userspace setting the > > > >> PMU, > > > >> events will stop counting if the VCPU is scheduled on the wrong CPU. > > > > > > > > Can't we fix the default behavior (in addition to the current fix) ? > > > > (Do we need to maintain the default behavior ??) > > > > > > Of course we do. This is a behaviour that has been exposed to userspace > > > for years, and *we don't break userspace*. > > > > I'm wondering if it might be better to have kvm_pmu_create_perf_event() > > set attr.type to pmu_id based on the current (physical) CPU by default > > on such heterogeneous systems (even if userspace don't explicitly > > specify pmu_id with the new API). Then, by setting the CPU affinity, > > the PMU in that environment can behave predictably even with existing > > userspace (or maybe this won't be helpful at all?). > > I think then you would end up with the possible mismatch between > kvm->arch.pmuver and the version of the PMU that is used for creating the > events. Yes, but, I would think we can have kvm_pmu_create_perf_event() set vcpu->arch.pmu.arm_pmu based on the current physical CPU when vcpu->arch.pmu.arm_pmu is null (then, the pmuver is handled as if KVM_ARM_VCPU_PMU_V3_SET_PMU was done implicitly). > Also, as VCPUs get migrated from one physical CPU to the other, the > semantics of the microarchitectural events change, even if the event ID is > the same. Yes, I understand. As mentioned, this can work only when the CPU affinity is set for vCPU threads appropriately (, which could be done even without changing userspace). Thanks, Reiji _______________________________________________ kvmarm mailing list kvmarm@xxxxxxxxxxxxxxxxxxxxx https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
