On Sat, Feb 03, 2024 at 05:30:48PM +0800, Zhao Liu wrote:
> Date: Sat, 3 Feb 2024 17:30:48 +0800
> From: Zhao Liu <zhao1.liu@xxxxxxxxxxxxxxx>
> Subject: [RFC 0/6] Intel Thread Director Virtualization Support in QEMU
> X-Mailer: git-send-email 2.34.1
>
> From: Zhao Liu <zhao1.liu@xxxxxxxxx>
>
> Hi list,
>
> This is our refreshed RFC to support our ITD virtualization patch
> series [1] in KVM, and bases on bd2e12310b18 ("Merge tag
> 'qga-pull-2024-01-30' of https://github.com/kostyanf14/qemu into
> staging").
>
> ITD is Intel's client specific feature to optimize scheduling on Intel
> hybrid platforms. Though this feature depends on hybrid topology
s/depends/doesn't depend/
Regards,
Zhao
> details, in our parctice on Win11 Guest, ITD works with hyrbid topolohy
> and CPU affinity can achieve the most performance improvement in Win11
> Guest (for example, on i9-13900K, up to 14%+ improvement on
> 3DMARK). More data or details, can be found in [1]. Thus, the ITD for
> Win11 is also a typical use case of hybrid topology.
>
>
> Welcome your feedback!
>
>
> 1. Background and Motivation
> ============================
>
> ITD allows the hardware to provide scheduling hints to the OS to help
> optimize scheduling performance, and under the Intel hybrid
> architecture, since Core and Atom have different capabilities
> (performance, energy effency, etc.), scheduling based on hardware
> hints can take full advantage of this hybrid architecture. This is also
> the most ideal scheduling model for intel hybrid architecture.
>
> Therefore, we want to virtualize the ITD feature so that ITD can benefit
> performance of the virtual machines on the hybrid machines as well.
>
> Currently, our ITD virtualization is a software virtualization solution.
>
>
> 2. Introduction to HFI and ITD
> ==============================
>
> Intel provides Hardware Feedback Interface (HFI) feature to allow
> hardware to provide guidance to the OS scheduler to perform optimal
> workload scheduling through a hardware feedback interface structure in
> memory [2]. This hfi structure is called HFI table.
>
> As for now, the guidance includes performance and energy enficency hints,
> and it could update via thermal interrupt as the actual operating
> conditions of the processor change during run time.
>
> And Intel Thread Director (ITD) feature extends the HFI to provide
> performance and energy efficiency data for advanced classes of
> instructions.
>
> The virtual HFI table is maintained in KVM, and for QEMU, we just need
> to handle HFI/ITD/HRESET (and their dependent features: ACPI, TM and
> PTS) related CPUIDs and MSRs.
>
>
> 3. Package level MSRs handling
> ==============================
>
> PTS, HFI and ITD are all have package level features, such as package
> level MSRs and package level HFI tables. But since KVM hasn't
> support msr-topology and it just handle these package-level MSRs and
> HFI table at VM level, in order to avoid potential contention problems
> caused by multiple virtual-packages, we restrict VMs to be able to
> enable PTC/HFI/ITD iff there's only 1 package (and only 1 die for
> ITD/HFI).
>
>
> 4. HFI/ITD related info in CPUID
> ================================
>
> KVM provides some basic HFI info in CPUID.0x06 leaf, which is associated
> with the virtual HFI table in KVM.
>
> QEMU should configure HFI table index for each vCPU. Here we set the HFI
> table index to vCPU index so that different vCPUs have different HFI
> entries to avoid unnecessary competition problems.
>
>
> 5. Compatibility issues
> =======================
>
> HFI is supported in both server (SPR) and client (ADL/RPL/MTL) platform
> products while ITD is the client specific feature.
>
> For client platform, ITD (with HFI) could be enabled in Guest to improve
> scheduling, but for server platform, HFI (without ITD) is only useful
> on Host and Guest doesn't need it.
>
> To simplify the enabling logic and avoid impacting the common topology
> of the Guest, we set PTS, HFI, and ITD as feature bits that are not
> automatically enabled.
>
> Only when the user actively specifies these features, QEMU will check
> and decide whether to enable them based on the topology constraints and
> the ITD constraints.
>
>
> 6. New option "enable-itd"
> ============================
>
> ITD-related features include PTS, HFI, ITD, and HRESET.
>
> To make it easier for users to enable ITD for Guest without specifying
> the above feature bits one by one, we provide a new option "enable-itd"
> to set the above feature bits for Guest all at once.
>
> "enable-itd" does not guarantee that ITD will be enabled for Guest.
> The success of enabling ITD for guest depends on topology constraints,
> platform support, etc., which are checked in QEMU.
>
>
> 7. Patch Summary
> ================
>
> Patch 1: Add support save/load for ACPI feature related thermal MSRs
> since ACPI feature CPUID has been added in QEMU.
> Patch 2: Add support for PTS (package) thermal MSRs and its CPUID
> Patch 3: Add support for HFI MSRs and its CPUID
> Patch 4: Add support ITD CPUID and MSR_IA32_HW_FEEDBACK_THREAD_CONFIG.
> Patch 5: Add support HRESET CPUID and MSR_IA32_HW_HRESET_ENABLE.
> Patch 6: Add "enable-itd" to help user set ITD related feature bits.
>
> # 8. References
>
> [1]: KVM RFC: [RFC 00/26] Intel Thread Director Virtualization
> https://lore.kernel.org/kvm/20240203091214.411862-1-zhao1.liu@xxxxxxxxxxxxxxx/T/#t
> [2]: SDM, vol. 3B, section 15.6 HARDWARE FEEDBACK INTERFACE AND INTEL
> THREAD DIRECTOR
>
> Thanks and Best Regards,
> Zhao
> ---
> Zhao Liu (2):
> target/i386: Add support for Intel Thread Director feature
> i386: Add a new property to set ITD related feature bits for Guest
>
> Zhuocheng Ding (4):
> target/i386: Add support for save/load of ACPI thermal MSRs
> target/i386: Add support for Package Thermal Management feature
> target/i386: Add support for Hardware Feedback Interface feature
> target/i386: Add support for HRESET feature
>
> target/i386/cpu.c | 108 ++++++++++++++++++++++++++++++++++++++++--
> target/i386/cpu.h | 37 +++++++++++++++
> target/i386/kvm/kvm.c | 84 ++++++++++++++++++++++++++++++++
> 3 files changed, 225 insertions(+), 4 deletions(-)
>
> --
> 2.34.1
>
