Hi Masahiro and Andrew, Top posting only for visibility. Would it make more sense to have this land via the Kbuild tree or -mm? The core of the series really touches Kbuild and I think the x86 stuff can just land with Acks, unless the -tip folks feel differently. I would like Rong to have a relatively clear path forward to mainline once the requisite review and testing has accomplished, which requires a shepherd :) Cheers, Nathan On Mon, Oct 14, 2024 at 02:33:34PM -0700, Rong Xu wrote: > Hi, > > This patch series is to integrate AutoFDO and Propeller support into > the Linux kernel. AutoFDO is a profile-guided optimization technique > that leverages hardware sampling to enhance binary performance. > Unlike Instrumentation-based FDO (iFDO), AutoFDO offers a user-friendly > and straightforward application process. While iFDO generally yields > superior profile quality and performance, our findings reveal that > AutoFDO achieves remarkable effectiveness, bringing performance close > to iFDO for benchmark applications. > > Propeller is a profile-guided, post-link optimizer that improves > the performance of large-scale applications compiled with LLVM. It > operates by relinking the binary based on an additional round of runtime > profiles, enabling precise optimizations that are not possible at > compile time. Similar to AutoFDO, Propeller too utilizes hardware > sampling to collect profiles and apply post-link optimizations to improve > the benchmark’s performance over and above AutoFDO. > > Our empirical data demonstrates significant performance improvements > with AutoFDO and Propeller, up to 10% on microbenchmarks and up to 5% > on large warehouse-scale benchmarks. This makes a strong case for their > inclusion as supported features in the upstream kernel. > > Background > > A significant fraction of fleet processing cycles (excluding idle time) > from data center workloads are attributable to the kernel. Ware-house > scale workloads maximize performance by optimizing the production kernel > using iFDO (a.k.a instrumented PGO, Profile Guided Optimization). > > iFDO can significantly enhance application performance but its use > within the kernel has raised concerns. AutoFDO is a variant of FDO that > uses the hardware’s Performance Monitoring Unit (PMU) to collect > profiling data. While AutoFDO typically yields smaller performance > gains than iFDO, it presents unique benefits for optimizing kernels. > > AutoFDO eliminates the need for instrumented kernels, allowing a single > optimized kernel to serve both execution and profile collection. It also > minimizes slowdown during profile collection, potentially yielding > higher-fidelity profiling, especially for time-sensitive code, compared > to iFDO. Additionally, AutoFDO profiles can be obtained from production > environments via the hardware’s PMU whereas iFDO profiles require > carefully curated load tests that are representative of real-world > traffic. > > AutoFDO facilitates profile collection across diverse targets. > Preliminary studies indicate significant variation in kernel hot spots > within Google’s infrastructure, suggesting potential performance gains > through target-specific kernel customization. > > Furthermore, other advanced compiler optimization techniques, including > ThinLTO and Propeller can be stacked on top of AutoFDO, similar to iFDO. > ThinLTO achieves better runtime performance through whole-program > analysis and cross module optimizations. The main difference between > traditional LTO and ThinLTO is that the latter is scalable in time and > memory. > > This patch series adds AutoFDO and Propeller support to the kernel. The > actual solution comes in six parts: > > [P 1] Add the build support for using AutoFDO in Clang > > Add the basic support for AutoFDO build and provide the > instructions for using AutoFDO. > > [P 2] Fix objtool for bogus warnings when -ffunction-sections is enabled > > [P 3] Change the subsection ordering when -ffunction-sections is enabled > > [P 4] Enable –ffunction-sections for the AutoFDO build > > [P 5] Enable Machine Function Split (MFS) optimization for AutoFDO > > [P 6] Add Propeller configuration to the kernel build > > Patch 1 provides basic AutoFDO build support. Patches 2 to 5 further > enhance the performance of AutoFDO builds and are functionally dependent > on Patch 1. Patch 6 enables support for Propeller and is dependent on > patch 2 and patch 3. > > Caveats > > AutoFDO is compatible with both GCC and Clang, but the patches in this > series are exclusively applicable to LLVM 17 or newer for AutoFDO and > LLVM 19 or newer for Propeller. For profile conversion, two different > tools could be used, llvm_profgen or create_llvm_prof. llvm_profgen > needs to be the LLVM 19 or newer, or just the LLVM trunk. Alternatively, > create_llvm_prof v0.30.1 or newer can be used instead of llvm-profgen. > > Additionally, the build is only supported on x86 platforms equipped > with PMU capabilities, such as LBR on Intel machines. More > specifically: > * Intel platforms: works on every platform that supports LBR; > we have tested on Skylake. > * AMD platforms: tested on AMD Zen3 with the BRS feature. The kernel > needs to be configured with “CONFIG_PERF_EVENTS_AMD_BRS=y", To > check, use > $ cat /proc/cpuinfo | grep “ brs” > For the AMD Zen4, AMD LBRV2 is supported, but we suspect a bug with > AMD LBRv2 implementation in Genoa which blocks the usage. > > Experiments and Results > > Experiments were conducted to compare the performance of AutoFDO-optimized > kernel images (version 6.9.x) against default builds.. The evaluation > encompassed both open source microbenchmarks and real-world production > services from Google and Meta. The selected microbenchmarks included Neper, > a network subsystem benchmark, and UnixBench which is a comprehensive suite > for assessing various kernel operations. > > For Neper, AutoFDO optimization resulted in a 6.1% increase in throughput > and a 10.6% reduction in latency. Unixbench saw a 2.2% improvement in its > index score under low system load and a 2.6% improvement under high system > load. > > For further details on the improvements observed in Google and Meta's > production services, please refer to the LLVM discourse post: > https://discourse.llvm.org/t/optimizing-the-linux-kernel-with-autofdo-including-thinlto-and-propeller/79108 ... > Rong Xu (6): > Add AutoFDO support for Clang build > objtool: Fix unreachable instruction warnings for weak funcitons > Change the symbols order when --ffuntion-sections is enabled > AutoFDO: Enable -ffunction-sections for the AutoFDO build > AutoFDO: Enable machine function split optimization for AutoFDO > Add Propeller configuration for kernel build. > > Documentation/dev-tools/autofdo.rst | 165 ++++++++++++++++++++++++++ > Documentation/dev-tools/index.rst | 2 + > Documentation/dev-tools/propeller.rst | 161 +++++++++++++++++++++++++ > MAINTAINERS | 14 +++ > Makefile | 2 + > arch/Kconfig | 42 +++++++ > arch/x86/Kconfig | 2 + > arch/x86/kernel/vmlinux.lds.S | 4 + > include/asm-generic/vmlinux.lds.h | 54 +++++++-- > scripts/Makefile.autofdo | 25 ++++ > scripts/Makefile.lib | 20 ++++ > scripts/Makefile.propeller | 28 +++++ > tools/objtool/check.c | 2 + > tools/objtool/elf.c | 15 ++- > 14 files changed, 524 insertions(+), 12 deletions(-) > create mode 100644 Documentation/dev-tools/autofdo.rst > create mode 100644 Documentation/dev-tools/propeller.rst > create mode 100644 scripts/Makefile.autofdo > create mode 100644 scripts/Makefile.propeller > > > base-commit: eb952c47d154ba2aac794b99c66c3c45eb4cc4ec > -- > 2.47.0.rc1.288.g06298d1525-goog >
