The following commit has been merged into the x86/mm branch of tip: Commit-ID: b64dfcde1ca9cb82e38e573753f0c0db8fb841c2 Gitweb: https://git.kernel.org/tip/b64dfcde1ca9cb82e38e573753f0c0db8fb841c2 Author: Borislav Petkov <bp@xxxxxxx> AuthorDate: Fri, 17 Dec 2021 16:48:29 +01:00 Committer: Borislav Petkov <bp@xxxxxxx> CommitterDate: Wed, 22 Dec 2021 11:51:20 +01:00 x86/mm: Prevent early boot triple-faults with instrumentation Commit in Fixes added a global TLB flush on the early boot path, after the kernel switches off of the trampoline page table. Compiler profiling options enabled with GCOV_PROFILE add additional measurement code on clang which needs to be initialized prior to use. The global flush in x86_64_start_kernel() happens before those initializations can happen, leading to accessing invalid memory. GCOV_PROFILE builds with gcc are still ok so this is clang-specific. The second issue this fixes is with KASAN: for a similar reason, kasan_early_init() needs to have happened before KASAN-instrumented functions are called. Therefore, reorder the flush to happen after the KASAN early init and prevent the compilers from adding profiling instrumentation to native_write_cr4(). Fixes: f154f290855b ("x86/mm/64: Flush global TLB on boot and AP bringup") Reported-by: "J. Bruce Fields" <bfields@xxxxxxxxxxxx> Reported-by: kernel test robot <oliver.sang@xxxxxxxxx> Signed-off-by: Borislav Petkov <bp@xxxxxxx> Tested-by: Carel Si <beibei.si@xxxxxxxxx> Tested-by: "J. Bruce Fields" <bfields@xxxxxxxxxxxx> Link: https://lore.kernel.org/r/20211209144141.GC25654@xsang-OptiPlex-9020 --- arch/x86/kernel/cpu/common.c | 2 +- arch/x86/kernel/head64.c | 16 ++++++++++++++-- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 0083464..79b3d67 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -384,7 +384,7 @@ set_register: } EXPORT_SYMBOL(native_write_cr0); -void native_write_cr4(unsigned long val) +void __no_profile native_write_cr4(unsigned long val) { unsigned long bits_changed = 0; diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 75acb60..f5e80a8 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -483,10 +483,12 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data) /* Kill off the identity-map trampoline */ reset_early_page_tables(); - __native_tlb_flush_global(native_read_cr4()); - clear_bss(); + /* + * This needs to happen *before* kasan_early_init() because latter maps stuff + * into that page. + */ clear_page(init_top_pgt); /* @@ -498,6 +500,16 @@ asmlinkage __visible void __init x86_64_start_kernel(char * real_mode_data) kasan_early_init(); + /* + * Flush global TLB entries which could be left over from the trampoline page + * table. + * + * This needs to happen *after* kasan_early_init() as KASAN-enabled .configs + * instrument native_write_cr4() so KASAN must be initialized for that + * instrumentation to work. + */ + __native_tlb_flush_global(this_cpu_read(cpu_tlbstate.cr4)); + idt_setup_early_handler(); copy_bootdata(__va(real_mode_data));
![]() |