On Tue, Feb 25, 2025 at 10:00:39PM -0500, Rik van Riel wrote: > Use broadcast TLB invalidation for kernel addresses when available. > > Remove the need to send IPIs for kernel TLB flushes. > > Signed-off-by: Rik van Riel <riel@xxxxxxxxxxx> > Tested-by: Manali Shukla <Manali.Shukla@xxxxxxx> > Tested-by: Brendan Jackman <jackmanb@xxxxxxxxxx> > Tested-by: Michael Kelley <mhklinux@xxxxxxxxxxx> > --- > arch/x86/mm/tlb.c | 32 ++++++++++++++++++++++++++++++-- > 1 file changed, 30 insertions(+), 2 deletions(-) Changes ontop: --- /tmp/current.patch 2025-02-28 22:39:33.236465716 +0100 +++ /tmp/0001-x86-mm-Use-INVLPGB-for-kernel-TLB-flushes.patch 2025-02-28 22:39:59.432310072 +0100 @@ -1,36 +1,43 @@ +From b97ae5e31069cd536b563df185de52d33a565077 Mon Sep 17 00:00:00 2001 From: Rik van Riel <riel@xxxxxxxxxxx> Date: Tue, 25 Feb 2025 22:00:39 -0500 -Subject: x86/mm: Use INVLPGB for kernel TLB flushes +Subject: [PATCH] x86/mm: Use INVLPGB for kernel TLB flushes Use broadcast TLB invalidation for kernel addresses when available. - Remove the need to send IPIs for kernel TLB flushes. + [ bp: Integrate dhansen's comments additions. ] + Signed-off-by: Rik van Riel <riel@xxxxxxxxxxx> Signed-off-by: Borislav Petkov (AMD) <bp@xxxxxxxxx> -Tested-by: Manali Shukla <Manali.Shukla@xxxxxxx> -Tested-by: Brendan Jackman <jackmanb@xxxxxxxxxx> -Tested-by: Michael Kelley <mhklinux@xxxxxxxxxxx> +Acked-by: Dave Hansen <dave.hansen@xxxxxxxxx> Link: https://lore.kernel.org/r/20250226030129.530345-5-riel@xxxxxxxxxxx --- - arch/x86/mm/tlb.c | 32 ++++++++++++++++++++++++++++++-- - 1 file changed, 30 insertions(+), 2 deletions(-) + arch/x86/mm/tlb.c | 39 +++++++++++++++++++++++++++++++++++++-- + 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c -index dbcb5c968ff9..f44a03bca41c 100644 +index dbcb5c968ff9..5c44b94ad5af 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c -@@ -1077,6 +1077,18 @@ void flush_tlb_all(void) +@@ -1077,6 +1077,25 @@ void flush_tlb_all(void) on_each_cpu(do_flush_tlb_all, NULL, 1); } ++/* Flush an arbitrarily large range of memory with INVLPGB. */ +static void invlpgb_kernel_range_flush(struct flush_tlb_info *info) +{ + unsigned long addr, nr; + + for (addr = info->start; addr < info->end; addr += nr << PAGE_SHIFT) { + nr = (info->end - addr) >> PAGE_SHIFT; ++ ++ /* ++ * INVLPGB has a limit on the size of ranges it can ++ * flush. Break up large flushes. ++ */ + nr = clamp_val(nr, 1, invlpgb_count_max); ++ + invlpgb_flush_addr_nosync(addr, nr); + } + __tlbsync(); -- Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette
