Re: [RFC PATCH v3 00/32] x86: enable FRED for x86-64

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



We tested the v3 FRED patch set on the Intel Simics® Simulator and a machine
with a 7th Intel(R) Core(TM) CPU.

Following are the LTP verion 20220121 test results on X86-64.
+--------------------------------------------+-------+-------+-------+-------+
|                   Config                   |  Pass |  Fail |  Skip |  Hang |
+--------------------------------------------+-------+-------+-------+-------+
|       the 7th Intel(R) Core(TM) CPU        |  1788 |  108  |  453  |   0   |
|                 6.2.0-rc7+                 |       |       |       |       |
+--------------------------------------------+-------+-------+-------+-------+
|       the 7th Intel(R) Core(TM) CPU        |  1788 |  108  |  453  |   0   |
|        6.2.0-rc7+ w/ FRED patch set        |       |       |       |       |
+--------------------------------------------+-------+-------+-------+-------+
|   Intel Simics® Simulator w/o FRED model   |  1797 |  111  |  440  |   1   |
|                 6.2.0-rc7+                 |       |       |       |       |
+--------------------------------------------+-------+-------+-------+-------+
|   Intel Simics® Simulator w/o FRED model   |  1797 |  111  |  440  |   1   |
|        6.2.0-rc7+ w/ FRED patch set        |       |       |       |       |
+--------------------------------------------+-------+-------+-------+-------+
|   Intel Simics® Simulator w/ FRED model    |  1797 |  111  |  440  |   1   |
|                 6.2.0-rc7+                 |       |       |       |       |
+--------------------------------------------+-------+-------+-------+-------+
|   Intel Simics® Simulator w/ FRED model    |  1797 |  111  |  440  |   1   |
| 6.2.0-rc7+ w/ FRED patch set FRED disabled |       |       |       |       |
+--------------------------------------------+-------+-------+-------+-------+
|   Intel Simics® Simulator w/ FRED model    |  1797 |  111  |  440  |   1   |
|        6.2.0-rc7+ w/ FRED patch set        |       |       |       |       |
+--------------------------------------------+-------+-------+-------+-------+
We were unable to identify any regression with the LTP tests. 

Following are the Kselftest results on X86-64.
+--------------------------------------------+-------+-------+-------+-------+
|                  Config                    |  Pass |  Fail |  Skip |  Hang |
+--------------------------------------------+-------+-------+-------+-------+
|       the 7th Intel(R) Core(TM) CPU        |  3224 |  454  |  591  |   5   |
|                 6.2.0-rc7+                 |       |       |       |       |
+--------------------------------------------+-------+-------+-------+-------+
|       the 7th Intel(R) Core(TM) CPU        |  3224 |  454  |  591  |   5   |
|        6.2.0-rc7+ w/ FRED patch set        |       |       |       |       |
+--------------------------------------------+-------+-------+-------+-------+
|   Intel Simics® Simulator w/o FRED model   |  1853 |  267  |  2143 |   11  |
|                 6.2.0-rc7+                 |       |       |       |       |
+--------------------------------------------+-------+-------+-------+-------+
|   Intel Simics® Simulator w/o FRED model   |  1853 |  267  |  2143 |   11  |
|        6.2.0-rc7+ w/ FRED patch set        |       |       |       |       |
+--------------------------------------------+-------+-------+-------+-------+
|   Intel Simics® Simulator w/ FRED model    |  1853 |  267  |  2143 |   11  |
|                 6.2.0-rc7+                 |       |       |       |       |
+--------------------------------------------+-------+-------+-------+-------+
|   Intel Simics® Simulator w/ FRED model    |  1853 |  267  |  2143 |   11  |
| 6.2.0-rc7+ w/ FRED patch set FRED disabled |       |       |       |       |
+--------------------------------------------+-------+-------+-------+-------+
|   Intel Simics® Simulator w/ FRED model    |  1847 |  272  |  2143 |   12  |
|        6.2.0-rc7+ w/ FRED patch set        |       |       |       |       |
+--------------------------------------------+-------+-------+-------+-------+
No regression was found on the bare metal, while the following are the
regressions on the Simics® Simulator.
+----------------------+----------------------+----------------------+
|                      |    Intel Simics®     |    Intel Simics®     |
|                      |  Simulator w/ FRED   |  Simulator w/ FRED   |
|      test name       |        model         |        model         |
|                      |  6.2.0-rc7+ w/ FRED  |  6.2.0-rc7+ w/ FRED  |
|                      |    patch set FRED    |      patch set       |
|                      |       disabled       |                      |
+----------------------+----------------------+----------------------+
|   kvm:hyperv_clock   |         PASS         |         FAIL         |
+----------------------+----------------------+----------------------+
| kvm:hyperv_features  |         PASS         |         FAIL         |
+----------------------+----------------------+----------------------+
| kvm:xen_vmcall_test  |         PASS         |         FAIL         |
+----------------------+----------------------+----------------------+
|    kvm:hyperv_ipi    |         PASS         |         FAIL         |
+----------------------+----------------------+----------------------+
| kvm:hyperv_tlb_flush |         PASS         |         FAIL         |
+----------------------+----------------------+----------------------+
|    bpf:test_progs    |         FAIL         |         HANG         |
+----------------------+----------------------+----------------------+
|  x86:sysret_rip_64   |         PASS         |         FAIL         |
+----------------------+----------------------+----------------------+
The test "x86:sysret_rip_64" is NOT a valid test on FRED, and there is a fix
from Ammar Faizi after we discussed it in the LKML. Other test failures are
under investigation.

FRED is a 64-bit only feature, however the 31st patch "x86/fred: allow dynamic
stack frame size" applies to X86-32, thus we ran Kselftest and LTP on X86-32 to
see if there is any regression, and we din't find any.

Following are the Kselftest results on X86-32.
+-------------------------------+---------+---------+---------+---------+
|             Config            |   Pass  |   Fail  |   Skip  |   Hang  |
+-------------------------------+---------+---------+---------+---------+
| the 5th Intel(R) Core(TM) CPU |   1361  |   267   |   301   |    3    |
|        FRED 6.2.0-rc7+        |         |         |         |         |
+-------------------------------+---------+---------+---------+---------+
| the 5th Intel(R) Core(TM) CPU |   1361  |   267   |   301   |    3    |
|  6.2.0-rc7+ w/ FRED patch set |         |         |         |         |
+-------------------------------+---------+---------+---------+---------+

Following are the LTP test results on X86-32.
+-------------------------------+---------+---------+---------+---------+
|             Config            |   Pass  |   Fail  |   Skip  |   Hang  |
+-------------------------------+---------+---------+---------+---------+
| the 5th Intel(R) Core(TM) CPU |   1849  |    81   |   419   |    0    |
|  6.2.0-rc7+ w/ FRED patch set |         |         |         |         |
+-------------------------------+---------+---------+---------+---------+
| the 5th Intel(R) Core(TM) CPU |   1849  |    81   |   419   |    0    |
|        FRED 6.2.0-rc7+        |         |         |         |         |
+-------------------------------+---------+---------+---------+---------+

Thanks
  --Shan

On Thu, 2023-02-23 at 23:01 -0800, Xin Li wrote:
> This patch set enables FRED for x86-64, and it's based on the previous LKGS
> patch set.
> 
> The Intel flexible return and event delivery (FRED) architecture defines
> simple
> new transitions that change privilege level (ring transitions).  The FRED
> architecture was designed with the following goals:
> 1) Improve overall performance and response time by replacing event delivery
> through the interrupt descriptor table (IDT event delivery) and event return
> by
> the IRET instruction with lower latency transitions.
> 2) Improve software robustness by ensuring that event delivery establishes the
> full supervisor context and that event return establishes the full user
> context.
> 
> The new transitions defined by the FRED architecture are FRED event delivery
> and,
> for returning from events, two FRED return instructions. FRED event delivery
> can
> effect a transition from ring 3 to ring 0, but it is used also to deliver
> events
> incident to ring 0. One FRED instruction (ERETU) effects a return from ring 0
> to
> ring 3, while the other (ERETS) returns while remaining in ring 0.
> 
> Search for the latest FRED spec in most search engines with this search
> pattern:
> 
>   site:intel.com FRED (flexible return and event delivery) specification
> 
> As of now there is no publicly avaiable CPU supporting FRED, thus the Intel
> Simics® Simulator is used as software development and testing vehicles. And
> it can be downloaded from:
>   
> https://www.intel.com/content/www/us/en/developer/articles/tool/simics-simulator.html
> 
> To enable FRED, Simics package 8112 QSP-CPU needs to be installed with CPU
> model configured as:
> 	$cpu_comp_class = "x86-experimental-fred"
> 
> Longer term, we should refactor common code shared by FRED and IDT into common
> shared files, and contain IDT code using a new config CONFIG_X86_IDT.
> 
> TODO: call external_interrupt() to reinject IRQ in KVM VMX.
> 
> Changes since v2:
> * Improve comments for changes in arch/x86/include/asm/idtentry.h.
> 
> Changes since v1:
> * call irqentry_nmi_{enter,exit}() in both IDT and FRED debug fault kernel
>   handler (Peter Zijlstra).
> * Initialize a FRED exception handler to fred_bad_event() instead of NULL
>   if no FRED handler defined for an exception vector (Peter Zijlstra).
> * Push calling irqentry_{enter,exit}() and instrumentation_{begin,end}()
>   down into individual FRED exception handlers, instead of in the dispatch
>   framework (Peter Zijlstra).
> 
> H. Peter Anvin (Intel) (24):
>   x86/traps: let common_interrupt() handle IRQ_MOVE_CLEANUP_VECTOR
>   x86/traps: add a system interrupt table for system interrupt dispatch
>   x86/traps: add external_interrupt() to dispatch external interrupts
>   x86/cpufeature: add the cpu feature bit for FRED
>   x86/opcode: add ERETU, ERETS instructions to x86-opcode-map
>   x86/objtool: teach objtool about ERETU and ERETS
>   x86/cpu: add X86_CR4_FRED macro
>   x86/fred: add Kconfig option for FRED (CONFIG_X86_FRED)
>   x86/fred: if CONFIG_X86_FRED is disabled, disable FRED support
>   x86/cpu: add MSR numbers for FRED configuration
>   x86/fred: header file with FRED definitions
>   x86/fred: make unions for the cs and ss fields in struct pt_regs
>   x86/fred: reserve space for the FRED stack frame
>   x86/fred: add a page fault entry stub for FRED
>   x86/fred: add a debug fault entry stub for FRED
>   x86/fred: add a NMI entry stub for FRED
>   x86/fred: FRED entry/exit and dispatch code
>   x86/fred: FRED initialization code
>   x86/fred: update MSR_IA32_FRED_RSP0 during task switch
>   x86/fred: let ret_from_fork() jmp to fred_exit_user when FRED is
>     enabled
>   x86/fred: disallow the swapgs instruction when FRED is enabled
>   x86/fred: no ESPFIX needed when FRED is enabled
>   x86/fred: allow single-step trap and NMI when starting a new thread
>   x86/fred: allow FRED systems to use interrupt vectors 0x10-0x1f
> 
> Xin Li (8):
>   x86/traps: add install_system_interrupt_handler()
>   x86/traps: export external_interrupt() for VMX IRQ reinjection
>   x86/fred: header file for event types
>   x86/fred: add a machine check entry stub for FRED
>   x86/fred: fixup fault on ERETU by jumping to fred_entrypoint_user
>   x86/ia32: do not modify the DPL bits for a null selector
>   x86/fred: allow dynamic stack frame size
>   x86/fred: disable FRED by default in its early stage
> 
>  .../admin-guide/kernel-parameters.txt         |   4 +
>  arch/x86/Kconfig                              |   9 +
>  arch/x86/entry/Makefile                       |   5 +-
>  arch/x86/entry/entry_32.S                     |   2 +-
>  arch/x86/entry/entry_64.S                     |   5 +
>  arch/x86/entry/entry_64_fred.S                |  59 +++++
>  arch/x86/entry/entry_fred.c                   | 234 ++++++++++++++++++
>  arch/x86/entry/vsyscall/vsyscall_64.c         |   2 +-
>  arch/x86/include/asm/cpufeatures.h            |   1 +
>  arch/x86/include/asm/disabled-features.h      |   8 +-
>  arch/x86/include/asm/entry-common.h           |   3 +
>  arch/x86/include/asm/event-type.h             |  17 ++
>  arch/x86/include/asm/extable_fixup_types.h    |   4 +-
>  arch/x86/include/asm/fred.h                   | 131 ++++++++++
>  arch/x86/include/asm/idtentry.h               |  76 +++++-
>  arch/x86/include/asm/irq.h                    |   5 +
>  arch/x86/include/asm/irq_vectors.h            |  15 +-
>  arch/x86/include/asm/msr-index.h              |  13 +-
>  arch/x86/include/asm/processor.h              |  12 +-
>  arch/x86/include/asm/ptrace.h                 |  36 ++-
>  arch/x86/include/asm/switch_to.h              |  10 +-
>  arch/x86/include/asm/thread_info.h            |  35 +--
>  arch/x86/include/asm/traps.h                  |  13 +
>  arch/x86/include/asm/vmx.h                    |  17 +-
>  arch/x86/include/uapi/asm/processor-flags.h   |   2 +
>  arch/x86/kernel/Makefile                      |   1 +
>  arch/x86/kernel/apic/apic.c                   |  11 +-
>  arch/x86/kernel/apic/vector.c                 |   8 +-
>  arch/x86/kernel/cpu/acrn.c                    |   7 +-
>  arch/x86/kernel/cpu/common.c                  |  88 ++++---
>  arch/x86/kernel/cpu/mce/core.c                |  11 +
>  arch/x86/kernel/cpu/mshyperv.c                |  22 +-
>  arch/x86/kernel/espfix_64.c                   |   8 +
>  arch/x86/kernel/fred.c                        |  73 ++++++
>  arch/x86/kernel/head_32.S                     |   3 +-
>  arch/x86/kernel/idt.c                         |   6 +-
>  arch/x86/kernel/irq.c                         |   6 +-
>  arch/x86/kernel/irqinit.c                     |   7 +-
>  arch/x86/kernel/kvm.c                         |   4 +-
>  arch/x86/kernel/nmi.c                         |  28 +++
>  arch/x86/kernel/process.c                     |   5 +
>  arch/x86/kernel/process_64.c                  |  21 +-
>  arch/x86/kernel/signal_32.c                   |  21 +-
>  arch/x86/kernel/traps.c                       | 175 +++++++++++--
>  arch/x86/lib/x86-opcode-map.txt               |   2 +-
>  arch/x86/mm/extable.c                         |  28 +++
>  arch/x86/mm/fault.c                           |  20 +-
>  drivers/xen/events/events_base.c              |   5 +-
>  kernel/fork.c                                 |   6 +
>  tools/arch/x86/include/asm/cpufeatures.h      |   1 +
>  .../arch/x86/include/asm/disabled-features.h  |   8 +-
>  tools/arch/x86/include/asm/msr-index.h        |  13 +-
>  tools/arch/x86/lib/x86-opcode-map.txt         |   2 +-
>  tools/objtool/arch/x86/decode.c               |  22 +-
>  54 files changed, 1156 insertions(+), 174 deletions(-)
>  create mode 100644 arch/x86/entry/entry_64_fred.S
>  create mode 100644 arch/x86/entry/entry_fred.c
>  create mode 100644 arch/x86/include/asm/event-type.h
>  create mode 100644 arch/x86/include/asm/fred.h
>  create mode 100644 arch/x86/kernel/fred.c
> 




[Index of Archives]     [KVM ARM]     [KVM ia64]     [KVM ppc]     [Virtualization Tools]     [Spice Development]     [Libvirt]     [Libvirt Users]     [Linux USB Devel]     [Linux Audio Users]     [Yosemite Questions]     [Linux Kernel]     [Linux SCSI]     [XFree86]

  Powered by Linux